 |
Claims  |
|
|
What is claimed is:
1. An image segmentation method for segmenting an object, which is not
present in a reference image, from an input image including the object
using the reference image, comprising:
the edge intensity distribution extraction step of extracting edge
intensity distributions in the reference image and input image;
the direction-classified line detection step of detecting, in each of a
plurality of blocks divided on an image screen, line components in terms
of predetermined directions for edges in the reference image and input
image on the basis of the extracted edge intensity distributions; and
the image region specifying step of specifying an existence range of the
object in the input image on the basis of a distribution of differences
between the detected line components in each of said directions between
the reference image and input image.
2. An image segmentation method for segmenting a predetermined image region
from an input image input from an external device such as an image input
device using a reference image, comprising:
the edge distribution extraction step of extracting edge distributions in
the input image and reference image;
the direction-classified line detection step of detecting, in each of a
plurality of blocks divided on an image screen, line distributions in
terms of predetermined directions in the input image and reference image
on the basis of the extracted edge distributions;
the singular edge extraction step of extracting a singular edge on the
basis of a difference between the detected line distributions in each of
said directions between the reference image and input image; and
the image extraction step of extracting the predetermined image region in
the input image on the basis of the extracted singular edge.
3. An image identification method for identifying an object, which is not
present in a reference image, in an input image including the object using
a standard model image representing a predetermined object and the
reference image, comprising:
the edge intensity distribution extraction step of extracting edge
intensity distributions in the reference image and input image;
the direction-classified line detection step of detecting, in each of a
plurality of blocks divided on an image screen, line components in terms
of predetermined directions for edges in the reference image and input
image on the basis of the extracted edge intensity distributions;
the auto-framing step of specifying an existence range of the object in the
input image on the basis of a distribution of differences between the
detected line components in each of said directions between the reference
image and input image; and
the model size estimation step of estimating a size with respect to the
standard model image on the basis of the specified existence range of the
object,
wherein a size of the standard model image is changed to the estimated
size, and thereafter, the object is identified on the basis of similarity
between the object image present in the existence range in the input image
and the size-changed standard model image.
4. An image segmentation method for extracting an object, which is not
present in a reference image, from an input image including the object
using the reference image, comprising:
the edge intensity distribution extraction step of extracting edge
intensity distributions in the reference image and input image;
the direction-classified line detection step of detecting, in each of a
plurality of blocks divided on an image screen, line components in terms
of predetermined directions for edges in the reference image and input
image on the basis of the extracted edge intensity distributions;
the auto-framing step of specifying an existence range of the object in the
input image on the basis of a distribution of differences between the
detected line components in each of said directions between the reference
image and input image; and
the extraction processing step of performing extraction processing of the
object within the specified existence range of the object.
5. An image segmentation method for segmenting an image region to be
extracted from an input image using a reference image that represents a
region approximating a remaining region excluding the image region,
comprising:
the edge intensity distribution extraction step of extracting edge
intensity distributions in the reference image and input image;
the direction-classified line detection step of detecting, in each of a
plurality of blocks divided on an image screen, line components in terms
of predetermined directions for edges in the reference image and input
image on the basis of the extracted edge intensity distributions;
the singular contour extraction step of extracting a singular contour
portion of the image region to be extracted on the basis of a distribution
of differences between the detected line components in each of said
directions in the reference image and input image; and
the image extraction step of extracting the image region to be extracted on
the basis of distribution data representing the extracted singular contour
portion.
6. The method according to claim 5, wherein the image extraction step
comprises:
the partial region extraction step of extracting a portion of the image
region to be extracted as a partial region from the input image;
the region growing step of performing region growing by thresholding
similarities between the extracted partial region as a seed and its
neighboring regions; and
the extraction step of extracting a region obtained by the region growth as
the image region to be extracted.
7. The method according to claim 5, wherein the singular contour extraction
step includes the step of extracting, as the singular contour portion,
edges in the input image which have different line labels in terms of
directions in identical neighboring regions of the input image and
reference image.
8. The method according to claim 5, wherein the singular contour extraction
step comprises:
the dominant line map extraction step of dividing the reference image into
a plurality of blocks and detecting dominant line direction components in
the blocks; and
the line direction comparison step of comparing a label assigned to each of
edges of the input image and the dominant direction line component in the
block to which that edge belongs, and
when the label assigned to the edge is different from the dominant
direction line component in the block to which that edge belongs, the edge
is extracted as the singular contour portion of the input image.
9. The method according to claim 5, wherein the image extraction step
comprises:
the partial region extraction step of binarizing a portion of the image
region to be extracted and extracting the binary data as mask data;
the smoothing step of smoothing the extracted mask data; and
the singular contour restoration step of restoring the singular contour
portion to mask data after the mask data is smoothed.
10. The method according to claim 5, wherein the singular contour
extraction step includes the step of detecting a occluding boundary line
serving as a boundary between the image region to be extracted, and the
remaining region, and determining the detected occluding boundary line as
the singular contour portion.
11. The method according to claim 10, wherein the singular contour
extraction step comprises:
the dominant line map extraction step of segmenting the reference image
into a plurality of blocks and detecting dominant line direction
components in the blocks,
a boundary point located in the vicinity of a boundary between a block
without any dominant direction line component, and a block with the
dominant direction line component among the blocks of the input image is
extracted, and
an edge of the input image located at a position closest to the boundary
point in a predetermined local region including the boundary point is
extracted as a portion of the occluding boundary line.
12. The method according to claim 6, wherein the region growing step
includes the step of controlling the region growing so that a region
growing direction from the edge substantially agrees with a label in units
of directions of that edge.
13. An image segmentation method for segmenting an image region to be
extracted from an input image using a reference image that represents a
region approximating a remaining region excluding the image region,
comprising:
the low-resolution image extraction step of extracting low-resolution image
portions in the input image and reference image;
the image matching step of performing matching corresponding points between
for the input image and reference image;
the dominant line map extraction step of segmenting the input image and
reference image into a plurality of blocks an detecting, in each of a
plurality of blocks divided on an image screen, dominant line direction
components in the blocks; and
the extraction step of extracting the image region on the basis of a degree
of mating between a label in each of said directions of each edge of the
input image and a label of the dominant line map of the reference image at
the edge position.
14. The method according to claim 13, wherein the image matching step
comprises:
the corresponding point extraction step of extracting corresponding points
between the reference image and input image;
the first transformation step of geometrically transforming one of the
input image and reference image on the basis of the extracted
corresponding points; and
the second transformation step of performing color correction of one of the
input image and reference image, so that corresponding pixels in regions
including the corresponding points have substantially equal gray levels
after the geometric transformation.
15. The method according to claim 14, wherein the geometric transformation
includes entire or local processing associated with at least one of a
translation, rotation, magnification transformation, and perspective
transformation.
16. An image segmentation apparatus for segmenting an image region to be
extracted from an input image using a reference image that represents a
region substantially equal to a remaining region excluding the image
region, comprising:
storage means for storing the reference image;
edge extraction means for extracting edge distributions in the input image
and reference image;
direction-classified line detection means for detecting, in each of a
plurality of blocks divided on an image screen, line distributions in
terms of directions in the input image and reference image on the basis of
the extracted edge distributions;
corresponding point extraction means for extracting corresponding point
information between the reference image and input image;
transformation means for geometrically transforming one of the input image
and reference image on the basis of the extracted corresponding point
information;
singular edge extraction means for extracting a singular edge on the basis
of a line distribution difference in each of said directions between the
geometrically transformed image, and the other image; and
segmentation means for segmenting the image region to be extracted from the
input image on the basis of the extracted singular edge.
17. The apparatus according to claim 16, wherein the geometric
transformation includes entire or local processing associated with at
least one of a translation, rotation, magnification transformation, and
perspective transformation.
18. An image identification apparatus for identifying an object in an input
image including the object which is not present in a reference image using
a standard model image representing a predetermined object and the
reference image, comprising:
edge intensity distribution extraction means for extracting edge intensity
distributions in the reference image and input image;
direction-classified line detection means for detecting, in each of a
plurality of blocks on an image screen, line components in terms of
predetermined directions for edges in the reference image and input image
on the basis of the extracted edge intensity distributions;
auto-framing means for specifying an existence range of the object in the
input image on the basis of a distribution of differences between the
detected line components in each of said directions between the reference
image and input image; and
model size estimation means for estimating a size with respect to the
standard model image on the basis of the specified existence range of the
object,
wherein a size of the standard model image is changed to the estimated
size, and thereafter, the object is identified on the basis of similarity
between the object image present in the existence range in the input image
and the size-changed standard model image. |
|
 |
|
 |
Claims |
|
|
|
Description |
|
|
BACKGROUND OF THE INVENTION
The present invention relates to an image segmentation method for
segmenting the image region to be extracted from an input image using a
reference image, an image identification method for identifying the image
region to be extracted from an input image using a reference image, an
image segmentation apparatus, an image processing apparatus and method for
extracting a specific image region from an image, and a storage medium
storing the image processing method.
As general techniques for implementing image extraction, a chromakey method
using a specific color background, a videomatte method for generating a
key signal by predetermined image processing (histogram processing,
difference processing, differential processing, contour enhancement,
contour tracing, and the like) (The Television Society Technical Report,
Vol. 12, pp. 29-34, 1988), and the like are known.
A technique for performing image extraction based on the difference from
the background image is a state-of-the-art one, and for example, Japanese
Patent Laid-Open No. 4-216181 discloses a technique for detecting or
extracting a target object in a plurality of specific regions in an image
by setting a mask image (specific processing region) in difference data
between the background image and the image to be processed. Furthermore,
Japanese Patent Publication No. 7-16250 discloses a technique that uses a
color model of the object to be extracted to implement image extraction by
obtaining the existence probability distribution of the object to be
extracted from color-converted data of an original image including
background and lightness difference data between the background image and
the original image.
In the difference method from the background image, the luminance level or
color component difference between the pixels of the background image and
the subject image is normally expressed by a predetermined evaluation
function, and the evaluation function is subjected to thresholding to
extract a region having a difference level equal to or higher than a
predetermined value. As the evaluation function, the correlation between
blocks having individual points as centers and a predetermined size
(Rosenfeld, A. and Kak, A. C., Digital Picture Processing (2nd ed.),
Academic Press, 1982), a normalized principal component features (Journal
of the Institute of Electronics, Information and Communication Engineers,
Vol.
J74-D-II, pp. 1731-1740), a weighted sum value of a standard deviation and
a difference value (Journal of the Television Society, Vol. 45, pp.
1270-1276, 1991), a local histogram distance associated with hue and
luminance level (Journal of the Television Society, Vol. 49, pp. 673-680,
1995), and the like are used.
As a technique for identifying or recognizing a specific object, the
following method is popularly used. That is, a model image or template
associated with that object is prepared in advance. An image region of the
object to be identified is separated from other regions, the size of the
image region to be identified is normalized or its position is fixed, or a
plurality of model images having different sizes are prepared. Scanning a
target image, the similarities between the model image or template and the
object to be identified are determined using a measure such as correlation
or the like.
The background difference method poses a problem when a partial region
which has a similar luminance, color, or pattern to the background image
is included in the object to be extracted. In this case, since no
difference in that region is estimated between the background image and
the input image including the object to be extracted, extraction or
detection errors take place. As a step against such problem, Japanese
Patent Laid-Open No. 8-44844 adopts a method of calculating the gradients
of both the background image and the input image, and taking logical OR of
the difference absolute value between the gradients and that of image
signals. On the other hand, Japanese Patent Laid-Open No. 8-212350 adopts
a method of performing thresholding by calculating a feature, the rate of
its change with respect to changes in pixel density of the input image of
which decreases when the background image has a middle pixel density, and
increases when the background image has a high or low pixel density.
However, in image extraction, the chromakey method is hard to use outdoors
due to serious background limitations such as a requirement for a specific
color background, and causes color omissions in the subject region having
the same color as the background. On the other hand, in the videomatte
method, since contour designation must be manually and accurately
performed in units of pixels, such operation requires much labor and
skill.
Furthermore, in the background difference method, the background is hard to
distinguish from the subject in a partial region of the subject similar to
the background, and this method does not normally allow image sensing
condition differences (e.g., the exposure condition, magnification,
illumination condition, focusing condition, view point position, and the
like) between the background image and input image. Especially, when
background obtained by removing the subject from the input image is
different from the background image, the tolerance to their difference is
considerably low even if they are similar to each other. In addition, it
is very hard to extract contour and three-dimensional shape details of the
subject while removing the noise influence.
Also, the background difference method requires distinct image
characteristic differences (e.g., pixel values and the like) between the
background image and the subject region everywhere, and it is hard to
apply the method to a general background. Even in the methods that take a
step against the partial region of the subject similar to the background
(Japanese Laid-Open Patent Nos. 8-44844 and 8-212350), for example, when
the rate of spatial change in pixel value is small, the subject is hardly
or insufficiently distinguished from the background. Hence, it is
difficult to stably maintain high extraction precision by automatic
processing in a practical use.
When the shadow of the object to be extracted is present in the image in
the vicinity of the object, it is hard for either method to extract the
image region to be extracted alone and to automatically remove the shadow.
Furthermore, in the method of identifying or recognizing a specific target
image, the segmentation processing from other regions in the
above-mentioned pre-processing normally constitutes in separable parts and
is complicated and hard to attain. On the recognition technique, also, it
is difficult to automatically normalize the size, position, and the like
since the size and position of the object to be recognized are not
detected in advance. Furthermore, the number of model images having
different sizes that can be prepared is limited due to storage capacity
limitations on the database, resulting in poor versatility.
SUMMARY OF THE INVENTION
It is, therefore, the first object of the present invention to provide an
image segmentation method, which can precisely and automatically detect
the existence range of an intruder in the input image, and can precisely
detect the existence range of the object to be extracted or recognized.
It is the second object of the present invention to provide an image
segmentation method which can stably extract an image from the input image
even when the reference image and the input image have differences
resulting from variations of the image sensing condition, illumination
condition, and the like therebetween.
It is the third object of the present invention to provide an image
identification method and apparatus, which can stably and automatically
recognize the object to be recognized without being influenced by any
differences of the image size and position of a subject, and are excellent
in terms of economy.
It is the fourth object of the present invention to provide an image
segmentation method which can perform high-speed image extraction while
removing the influences of noise such as shading.
It is the fifth object of the present invention to provide an image
segmentation method and apparatus, which can perform image extraction with
high tolerance to differences between the reference image and the
background portion of the input image.
It is the sixth object of the present invention to provide an image
processing apparatus and method, and a storage medium that stores the
method, which can precisely extract a specific image region from a main
image which includes a specific image region to be extracted and a sub
image which does not include any specific image region to be extracted.
It is the seventh object of the present invention to provide an image
processing apparatus and method, and a storage medium that stores the
method, which can precisely extract a specific image region even when the
specific image region to be extracted includes a region similar to an
image which is not to be extracted.
It is the eighth object of the present invention to provide an image
processing apparatus and method, and a storage medium that stores the
method, which can precisely extract a specific image region even when an
image which is similar to the specific image to be extracted but is not to
be extracted is present in the vicinity of the specific image region to be
extracted.
It is the ninth object of the present invention to provide an image
processing apparatus and method, and a storage medium that stores the
method, which can precisely extract a specific image region even when the
specific image region to be extracted includes many regions similar to an
image which is not to be extracted.
It is the tenth object of the present invention to provide an image
processing apparatus and method, and a storage medium that stores the
method, which can precisely extract a specific image region even when the
specific image region to be extracted includes hole regions.
It is the eleventh object of the present invention to provide an image
processing apparatus and method, and a storage medium that stores the
method, which can precisely extract a specific image region at high speed
by inputting a rough shape of the specific region to be extracted in
advance.
In order to solve the above problems and to achieve the above objects, the
present invention comprises the following arrangement.
More specifically, there is provided an image segmentation method for
segmenting an object, which is not present in a reference image, from an
input image including the object using the reference image, comprising:
the edge intensity distribution extraction step of extracting edge
intensity distributions in the reference image and input image;
the direction-classified line detection step of detecting line components
in terms of predetermined directions for edges in the reference image and
input image on the basis of the extracted edge intensity distributions;
and
the image region specifying step of specifying an existence range of the
object in the input image on the basis of a distribution of differences
between the detected line components in terms of directions between the
reference image and input image.
This image segmentation method can precisely and automatically detect the
existence region of an intruder object in the input image, and can
precisely detect the existence range of the object to be extracted or
recognized.
Since the existence range of an object in the input image is specified on
the basis of the distribution of line components in terms of directions,
the existence region can be detected more accurately by removing the
influences of, e.g., shading.
In order to achieve the above objects, according to the present invention,
there is provided an image segmentation method using a reference for
segmenting a predetermined image region from an input image input from an
external device such as an image input device, comprising:
the edge distribution extraction step of extracting edge distributions in
the input image and reference image;
the direction-classified line detection step of detecting line
distributions in terms of predetermined directions in the input image and
reference image on the basis of the extracted edge distributions;
the singular edge extraction step of extracting a singular edge on the
basis of a difference between the detected line distributions in units of
directions between the reference image and input image; and
the image extraction step of extracting the predetermined image region in
the input image on the basis of the extracted singular edge.
This image segmentation method can stably extract an image from the input
image even when the reference image and the input image have differences
resulting from variations of the image sensing condition, illumination
condition, and the like therebetween.
In order to achieve the above objects, according to the present invention,
there is provided an image identification method for identifying an
object, which is not present in a reference image, in an input image
including the object using a standard model image representing a
predetermined object and the reference image, comprising:
the edge intensity distribution extraction step of extracting edge
intensity distributions in the reference image and input image;
the direction-classified line detection step of detecting line components
in terms of predetermined directions for edges in the reference image and
input image on the basis of the extracted edge intensity distributions;
the auto-framing step of specifying an existence range of the object in the
input image on the basis of a distribution of differences between the
detected line components in units of directions between the reference
image and input image; and
the model size estimation step of estimating a size with respect to the
standard model image on the basis of the specified existence range of the
object,
wherein a size of the standard model image is changed to the estimated
size, and thereafter, the object is identified on the basis of similarity
between the object image present in the existence range in the input image
and the size-changed standard model image.
This image identification method allows stable and automatic recognition of
the object to be recognized without being influenced by the image size and
position differences of a subject. Also, since standard model images
having different sizes need not be stored, an economical advantage can
also be expected.
In order to achieve the above objects, according to the present invention,
there is provided an image segmentation method for segmenting an object,
which is not present in a reference image, from an input image including
the object using the reference image, comprising:
the edge intensity distribution extraction step of extracting edge
intensity distributions in the reference image and input image;
the direction-classified line detection step of detecting line components
in terms of predetermined directions for edges in the reference image and
input image on the basis of the extracted edge intensity distributions;
the auto-framing step of specifying an existence range of the object in the
input image on the basis of a distribution of differences between the
detected line components in terms of directions between the reference
image and input image; and
the extraction processing step of performing extraction processing of the
object within the specified existence range of the object.
This image segmentation method can extract an image at high speed while
removing the influences of noise such as shading.
In order to achieve the above objects, according to the present invention,
there is provided an image segmentation method for segmenting an image
region to be extracted from an input image using a reference image that
represents a region approximating a remaining region excluding the image
region to be extracted, comprising:
the edge intensity distribution extraction step of extracting edge
intensity distributions in the reference image and input image;
the direction-classified line detection step of detecting line components
in terms of predetermined directions for edges in the reference image and
input image on the basis of the extracted edge intensity distributions;
the singular contour extraction step of extracting a singular contour
portion of the image region to be extracted on the basis of a distribution
of the detected line components in terms of directions in the reference
image and input image; and the image extraction step of extracting the
image region to be extracted on the basis of distribution data
representing the extracted singular contour portion.
This image segmentation method can extract only a contour inherent to a
subject by absorbing differences such as a positional offset, rotational
offset, distortion, and the like between the reference image and the
background portion of the input image if they are present, and can achieve
image extraction with high tolerance to the differences between the
reference image and the background portion of the input image.
In order to achieve the above objects, according to the present invention,
there is provided an image segmentation method for segmenting an image
region to be extracted from an input image using a reference image that
represents a region approximating a remaining region excluding the image
region, comprising:
the low-resolution image extraction step of extracting low-resolution image
portions in the input image and reference image;
the image matching step of performing matching corresponding points between
the input image and reference image;
the dominant line map extraction step of segmenting the input image and
reference image into a plurality of blocks and detecting dominant line
direction components in the blocks; and
the extraction step of extracting the image region on the basis of a degree
of matching between a label in terms of directions of each edge of the
input image and a label of the dominant line map of the reference image at
the edge position.
This image segmentation method can achieve image extraction with high
tolerance to differences between the reference image and the background
portion of the input image. For example, even when the background portion
of the input image and the corresponding region in the reference image are
substantially different scenes but have high similarity, or when the input
image and the reference image have different photographing conditions or
photographing means, image extraction can be performed with high
precision.
In order to achieve the above objects, according to the present invention,
there is provided an image segmentation apparatus for segmenting an image
region to be extracted from an input image using a reference image that
represents a region approximating a remaining region excluding the image
region, comprising:
storage means for storing the reference image;
edge extraction means for extracting edge distributions in the input image
and reference image;
direction-classified line detection means for detecting line distributions
in terms of directions in the input image and reference image on the basis
of the extracted edge distributions;
corresponding point extraction means for extracting corresponding point
information between the reference image and input image;
transformation means for geometrically transforming one of the input image
and reference image on the basis of the extracted corresponding point
information;
singular edge extraction means for extracting a singular edge on the basis
of a line distribution difference in units of directions between the
geometrically transformed image, and the other image; and
segmentation means for segmenting the image region to be extracted from the
input image on the basis of the extracted singular edge.
This image segmentation apparatus can extract only a contour inherent to a
subject by absorbing differences such as a positional offset, rotational
offset, distortion, and the like between the reference image and the
background portion of the input image if they are present, and can achieve
image extraction with high tolerance to the differences between the
reference image and the background portion of the input image.
In order to achieve the above objects, according to the present invention,
there is provided an image identification apparatus for identifying an
object in an input image including the object which is not present in a
reference image using a standard model image representing a predetermined
object and the reference image, comprising:
edge intensity distribution extraction means for extracting edge intensity
distributions in the reference image and input image;
direction-classified line detection means for detecting line components in
terms of predetermined directions for edges in the reference image and
input image on the basis of the extracted edge intensity distributions;
auto-framing means for specifying an existence range of the object in the
input image on the basis of a distribution of differences between the
detected line components in terms of directions between the reference
image and input image; and
model size estimation means for estimating a size with respect to the
standard model image on the basis of the specified existence range of the
object,
wherein a size of the standard model image is changed to the estimated
size, and thereafter, the object is identified on the basis of similarity
between the object image present in the existence range in the input image
and the size-changed standard model image.
This image identification apparatus can stably and automatically recognize
the object to be recognized without being influenced by the image size and
position differences of a subject since it changes the size of a standard
model image to the required size, and thereafter, identifies an object on
the basis of the similarity between an object image present in the
existence range in the input image and the standard model image, the size
of which has changed. Furthermore, since standard model images having
different sizes need not be stored, an economical advantage can also be
expected.
In order to achieve the above objects, according to the present invention,
there is provided an image processing apparatus comprising:
image input means for | | |
