Image capturing and processing apparatus and image capturing and processing method

What is claimed is:

1. An image capturing and processing apparatus comprising:

image pickup means;

first image memorizing means in which an object image from a predetermined viewing point position captured by said image pickup means is memorized;

three-dimensional shape model memorizing means for memorizing a reference three-dimensional shape model regarding the object image captured by said image pickup means, and producing a reference object image from a viewing point position nearest to the object image captured by said image pickup means, on the basis of said reference three-dimensional shape model;

second image memorizing means in which the reference object image produced by said three-dimensional shape model memorizing means is memorized;

different portion extracting means for extracting the different portion between the object image memorized in said first image memorizing means and the reference object image memorized in said second image memorizing means; and

shape model correcting means for correcting the reference three-dimensional shape model memorized in said three-dimensional shape model memorizing means, on the basis of the different portion extracted by said different portion extracting means.

2. An image capturing and processing apparatus according to claim 1, wherein

a plurality of object images from different viewing point positions are memorized in said first image memorizing means,

a plurality of reference object images corresponding to the plurality of object images from said different viewing point positions are memorized in said second image memorizing means,

said different portion extracting means extracts the different portions between the mutually corresponding ones of the object images memorized in said first image memorizing means and the reference object images memorized in said second image memorizing means, and

said shape model correcting means corrects the reference three-dimensional shape model memorized in said three-dimensional shape model memorizing means, on the basis of the respective different portions extracted by said different portion extracting means.

3. An image capturing and processing apparatus according to claim 2, wherein the reference three-dimensional shape model memorized in said three-dimensional shape model memorizing means is a mesh-like wire frame model divided into predetermined shapes, and

said different portion extracting means extracts the different portion between the object image and the reference object image on the basis of the central coordinates of the respective elements of said wire frame model divided into mesh-like shapes and the normal vector at said central coordinates.

4. An image capturing and processing apparatus according to claim 2, wherein after the correction of the reference three-dimensional shape model by said shape model correcting means is effected, said three-dimensional shape model memorizing means produces the reference object image on the basis of said corrected reference three-dimensional shape model, and

said shape model correcting means repetitively effects the correction of the reference three-dimensional shape model until the different portion between the reference object image based on the reference three-dimensional shape model after correction and the object image extracted by said different portion extracting means becomes a predetermined threshold value or less.

5. An image capturing and processing apparatus according to claim 4, wherein

said different portion extracting means is one which extracts the difference regarding structural information for correcting the reference three-dimensional shape model and the difference regarding texture information for correcting the reference object image produced by said three-dimensional shape model memorizing means, and

has texture correcting means for correcting the reference object image after the correction of the reference three-dimensional shape model produced by said three-dimensional shape model memorizing means, on the basis of the difference regarding the texture information extracted by said different portion extracting means.

6. An image capturing and processing apparatus according to claim 1, wherein the correction of the reference three-dimensional shape model effected in said shape model correcting means is effected by the use of an affine transformation coefficient, and said shape model correcting means effects the correction of the reference three-dimensional shape model by the use of such an affine transformation coefficient that the different portion extracted by said different portion extracting means becomes minimum.

7. An image capturing and processing apparatus according to claim 1, wherein the reference three-dimensional shape model memorized in said three-dimensional shape model memorizing means is a mesh-like wire frame model divided into predetermined shapes, and

said different portion extracting means extracts the different portion between the object image and the reference object image on the basis of the central coordinates of the respective elements of said wire frame model divided into mesh-like shapes and the normal vector at said central coordinates.

8. An image capturing and processing apparatus according to claim 1, wherein after the correction of the reference three-dimensional shape model by said shape model correcting means is effected, said three-dimensional shape model memorizing means produces the reference object image on the basis of said corrected reference three-dimensional shape model, and

said shape model correcting means repetitively effects the correction of the reference three-dimensional shape model until the different portion between the reference object image based on the reference three-dimensional shape model after correction and the object image extracted by said different portion extracting means becomes a predetermined threshold value or less.

9. An image capturing and processing apparatus according to claim 8, wherein

said different portion extracting means is one which extracts the difference regarding structural information for correcting the reference three-dimensional shape model and the difference regarding texture information for correcting the reference object image produced by said three-dimensional shape model memorizing means, and

has texture correcting means for correcting the reference object image after the correction of the reference three-dimensional shape model produced by said three-dimensional shape model memorizing means, on the basis of the difference regarding the texture information extracted by said different portion extracting means.

10. An image capturing and processing apparatus comprising:

image pickup means;

image memorizing means in which an object image from a predetermined viewing point position captured by said image pickup means is memorized;

three-dimensional shape model memorizing means in which a reference three-dimensional shape model regarding the object image captured by said image pickup means is memorized;

contour image extracting means for extracting a contour image regarding a reference object image from a viewing point position nearest to the object image captured by said image pickup means, on the basis of the reference three-dimensional shape model memorized in said three-dimensional shape model memorizing means;

initial contour setting means for setting an initial contour so that the position and size of the contour image regarding the reference object image extracted by said contour image extracting means may subsume the image area of the object image memorized in said image memorizing means;

active contour processing means for producing an contour image regarding the object image memorized in said image memorizing means, on the basis of the contour of the contour image of which the initial contour has been set by said initial contour setting means;

difference extracting means for extracting the difference between the contour image regarding the object image produced by said active contour processing means and the contour image regarding the reference object image of which the initial contour has been set by said initial contour setting means; and

shape model correcting means for correcting the reference three-dimensional shape model memorized in said three-dimensional shape model memorizing means, on the basis of the difference extracted by said difference extracting means.

11. An image capturing and processing apparatus according to claim 10, wherein said contour image extracting means extracts the contour image regarding such a reference object image that the difference extracted by said difference extracting means becomes minimum.

12. An image capturing and processing method of restituting a three-dimensional shape model regarding an object on the basis of an object image captured by image pickup means comprising:

the first step of memorizing a reference three-dimensional shape model regarding the object image-captured;

the second step of producing a reference object image corresponding to the object image captured by said image pickup means, on the basis of the reference three-dimensional shape model memorized at said first step;

the third step of extracting the difference between the reference object image produced at said second step and the object image captured by said image pickup means; and

the fourth step of correcting the reference three-dimensional shape model regarding the object memorized at said first step, on the basis of the difference extracted at said third step, and restituting the shape model of the object.

13. An image capturing and processing method according to claim 12, wherein at said third step, the differences between mutually corresponding ones of a plurality of object images from different viewing point positions captured by said image pickup means and a plurality of reference object images from different viewing point positions corresponding to said plurality of object images are extracted, and

at said fourth step, the reference three-dimensional shape model regarding the object is corrected on the basis of the difference extracted at said third step.

14. An image capturing and processing method of restituting a reference three-dimensional shape model regarding an object on the basis of an object image captured by image pickup means comprising:

the first step of memorizing a reference three-dimensional shape model regarding the object image captured;

the second step of extracting the contour image of a reference object image corresponding to the object image captured by said image pickup means, on the basis of the reference three-dimensional shape model memorized at said first step;

the third step of setting an initial contour so that the position and size of the contour image regarding the reference object image extracted at said second step may subsume the image area of the object image captured by said image pickup means;

the fourth step of producing a contour image regarding the object image captured by said image pickup means, on the basis of the contour image regarding the reference object image of which the initial contour has been set at said third step;

the fifth step of extracting the difference between the contour image regarding the object image produced at said fourth step and the contour image regarding the reference object image of which the initial contour has been set at said third step; and

the sixth step of correcting the reference three-dimensional shape model regarding the object memorized at said first step, on the basis of the difference extracted at said first step, and restituting the shape model of the object.

Description Submit all comments and votes

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an image capturing and processing apparatus and an image capturing and processing method for recovering a three-dimensional shape model regarding an object on the basis of the image of the object.

2. Related Background Art

As the technique of recovering a three-dimensional shape model regarding an object on the basis of the image of the object obtained by image pickup, there are the following systems and method disclosed, for example, in Japanese Laid-Open Patent Application No. 4-306782, Japanese Patent Publication No. 5-64393 and Japanese Laid-Open Patent Application No. 4-86957, respectively.

(1) The System Disclosed in Japanese Laid-Open Patent Application No. 4-306782

By the use of the slit light projection method (or the light-section method) of applying slit light to an object and picking it up by a camera, extracting characteristic points from the image of the object obtained by this image pickup, and calculating the three-dimensional structural information of the object on the basis of the extracted characteristic points, three-dimensional coordinates data at each characteristic point of the image of the object are calculated. On the basis of the three-dimensional coordinates data at each characteristic point obtained thereby, a basic three-dimensional wire frame registered in a data base is adjusted to the image of the object to thereby effect the recovery of a three-dimensional shape model.

(2) The Method Disclosed in Japanese Patent Publication No. 5-64393

On the basis of a two-dimensional photographed image obtained by photographing an object from a plurality of visual axis directions, an object existing area reflecting the shape of the object as it is seen from each visual axis direction is described as an aggregate of three-dimensional picture elements in a three-dimensional digital image space, and an aggregate of three-dimensional picture elements that exist in common in the three-dimensional space as it is seen from all directions is found as the three-dimensional shape of the object in the three-dimensional digital image to thereby effect the recovery of a three-dimensional shape model.

(3) The System Disclosed in Japanese Laid-Open Patent Application No. 4-86957

This is a system in which a near infrared reflection tape or a marker or the like of a hue differing from the background is imparted to a characteristic point of an object and the object image is captured from a plurality of directions by an image pickup device and from the obtained image, the three-dimensional coordinates of the characteristic points or lines are extracted based on a standard three-dimensional model. Concretely, the three-dimensional coordinates are collated with and interpolated with the result of measured each grid point of a three-dimensional grid model which is the reference from each direction to thereby effect the calibration of the position of the characteristic point, and the conversion look-up table of a fixed camera coordinates system and a reference coordinates system is prepared, whereafter the object with markers or the like is captured from a plurality of directions and three-dimensional coordinates of the characteristic point are obtained to thereby effect the recovery of the three-dimensional shape of the object.

However, the systems and method disclosed in the above-mentioned publications suffer from the following problems, respectively.

In the systems disclosed in Japanese Laid-Open Patent Application No. 4-306782 and Japanese Laid-Open Patent Application No. 4-86957, a special device such as slit light projecting means or means for imparting a marker is necessary as the means for directly measuring the three-dimensional shape of the object. This poses a problem in achieving the downsizing and low cost of the apparatus. Further, outside the area where the slit light is projected or the marker is imparted, the three-dimensional coordinates need be found by a process such as interpolation, and this leads to the problem that has the process complicated.

In the method disclosed in Japanese Patent Publication No. 5-64393, it is necessary to take the visual axis direction equally in every azimuth, and this leads to the problem that much time is required for processing. There arises a further problem that the recovery of a shape model accurately reflecting local three-dimensional structure is difficult.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide an image capturing and processing apparatus which can simply reconstruct a three-dimensional shape model from an object image without requiring any special device such as slit light projecting means or means for imparting a marker and any complicated process such as an interpolating process. It is a further object of the present invention to provide an image capturing and processing apparatus which can accurately reflect local three-dimensional structure for the recovery of a three-dimensional shape model.

The image capturing and processing apparatus of the present invention is characterized by image pickup means, first image memorizing means in which an object image from a predetermined viewing point position captured by the image pickup means is memorized, three-dimensional shape model memorizing means for memorizing a reference three-dimensional shape model regarding the object image captured by the image pickup means, and producing a reference object image from a viewing point position nearest to the object image captured by the image pickup means, on the basis of the reference three-dimensional shape model, second image memorizing means in which the reference object image produced by the three-dimensional shape model memorizing means is memorized, difference extracting means for extracting the difference between the object image memorized in the first image memorizing means and the reference object image memorized in the second image memorizing means, and shape model correcting means for correcting the reference three-dimensional shape model memorized in the three-dimensional shape model memorizing means, on the basis of the difference extracted by the difference extracting means.

In this case, a plurality of object images from different viewing point positions may be memorized in the first image memorizing means, a plurality of reference object images corresponding to the plurality of object images from the different viewing point positions may be memorized in the second image memorizing means, the difference extracting means may extract the difference between the mutually corresponding ones of the object images memorized in the first image memorizing means and the reference object images memorized in the second image memorizing means, and

the shape model correcting means may correct the reference three-dimensional shape model memorized in the three-dimensional shape model memorizing means, on the basis of the respective difference extracted by the different portion extracting means.

Further, the correction of the reference three-dimensional shape model effected in the shape model correcting means may be effected by the use of an affine transformation coefficient, and the shape model correcting means may effect the correction of the reference three-dimensional shape model by the use of such an affine transformation coefficient that the different portion extracted by the difference extracting means becomes minimum.

In the above-described image capturing and processing apparatus, design may be made such that the reference three-dimensional shape model memorized in the three-dimensional shape model memorizing means is a mesh-like wire frame model divided into predetermined shapes, and that the different portion extracting means extracts the different portion between the object image and the reference object image on the basis of the representative coordinates of the respective elements of the wire frame model divided into mesh-like shapes and the normal vector at the corresponding coordinate point.

Further, design may be made such that after the correction of the reference three-dimensional shape model by the shape model correcting means is effected, the three-dimensional shape model memorizing means produces the reference object image on the basis of the corrected reference three-dimensional shape model, and

that the shape model correcting means repetitively effects the correction of the reference three-dimensional shape model until the different portion between the reference object image based on the reference three-dimensional shape model after the correction and the object image extracted by the different portion extracting means becomes a predetermined threshold value or less.

In this case, the different portion extracting means may be one which extracts the difference regarding structural information for correcting the reference three-dimensional shape model and the difference regarding texture information for correcting the reference object image produced by the three-dimensional shape model memorizing means, and

which has texture correcting means for correcting the reference object image after the correction of the reference three-dimensional shape model produced by the three-dimensional shape model memorizing means, on the basis of the difference regarding the texture information extracted by the different portion extracting means.

Also, the image capturing and processing apparatus of the present invention is characterized by image pickup means, image memorizing means in which an object image from a predetermined viewing point position captured by the image pickup means, three-dimensional shape model memorizing means in which a reference three-dimensional shape model regarding the object captured by the image pickup means is memorized, contour image extracting means for extracting a boundary contour image regarding a reference object image from a viewing point position nearest to the object image captured by the image pickup means, on the basis of the reference three-dimensional shape model memorized in the three-dimensional shape model memorizing means, initial contour setting means for setting an initial contour so that the position and size of the contour image regarding the reference object image extracted by the contour image extracting means may subsume the image area of the object image memorized in the image memorizing means, active contour processing means for producing a contour image regarding the object image memorized in the image memorizing means, on the basis of the contour of the contour image of which the initial contour has been set by the initial contour setting means, difference extracting means for extracting the difference between the contour image regarding the object image produced by the active contour processing means and the contour image regarding the reference object image of which the initial contour has been set by the initial contour setting means; and

shape model correcting means for correcting the reference three-dimensional shape model memorized in the three-dimensional shape model memorizing means, on the basis of the difference extracted by the difference extracting means.

In this case, the contour image extracting means may be one which extracts the contour image regarding such a reference object image that the difference extracted by the difference extracting means becomes minimum.

Also, the image capturing and processing method of the present invention is an image capturing and processing method of recovering a three-dimensional shape model regarding an object on the basis of an object image captured by image pickup means, characterized by the first step of memorizing a reference three-dimensional shape model regarding the object image-captured, the second step of producing a reference object image corresponding to the object image captured by the image pickup means, on the basis of the reference three-dimensional shape model memorized at the first step, the third step of extracting the difference between the reference object image produced at the second step and the object image captured by the image pickup means; and

the fourth step of correcting the reference three-dimensional shape model regarding the object memorized at the first step, on the basis of the difference extracted at the third step, and recovering the shape model of the object.

In this case, design may be made such that at the third step, the differences between mutually corresponding ones of a plurality of object images from different viewing point positions captured by the image pickup means and a plurality of reference object images from different viewing point positions corresponding to the plurality of object images are extracted, and that at the fourth step, the reference three-dimensional shape model regarding the object is corrected on the basis of the differences extracted at the third step.

Also, the image capturing and processing method of the present invention is an image capturing and processing method of recovering a reference three-dimensional shape model regarding an object on the basis of an object image contured by image pickup means, characterized by, the first step of memorizing a reference three-dimensional shape model regarding the object image captured,

the second step of extracting the contour image of a reference object image corresponding to the object image captured by the image pickup means, on the basis of the reference three-dimensional shape model memorized at the first step; the third step of setting an initial contour so that the position and size of the contour image regarding the reference object image extracted at the second step may subsume the image area of the object image captured by the image pickup means;

the fourth step of producing a contour image regarding the object image captured by the image pickup means, on the basis of the contour image regarding the reference object image of which the initial contour has been set at the third step; the fifth step of extracting the difference between the contour image regarding the object image produced at the fourth step and the contour image regarding the reference object image of which the initial contour has been set at the third step; and the sixth step of correcting the reference three-dimensional shape model regarding the object memorized at the first step, on the basis of the difference extracted at the fifth step, and recovering the shape model object.

In the image processing apparatus and processing method of the present invention, the typical shape model of an object, for example, if the object is a particular man's head, the standard shape model of the man's head, is memorized as a reference three-dimensional shape model. On the basis of the memorized reference three-dimensional shape model, a reference object image corresponding to the object image captured by the image pickup means, i.e., a reference object image of which the viewing point position (visual axis direction) and the image size are the same as those of the object image is produced. The difference between the thus produced reference object image and the object image reflects the difference in similarity of shape between the three-dimensional shapes of the object and the reference three-dimensional shape model and the difference between the two-dimensional shapes or dispositions of various portions (such as the eyes and the mouth). Accordingly, if the reference three-dimensional shape model of the object is corrected on the basis of the difference between the reference object image and the object image, the difference between the three-dimensional shapes of the object and the reference three-dimensional shape model and the difference between the two-dimensional shapes or the dispositions of the various portions are reflected in the corrected reference three-dimensional shape model. By thus effecting the correction of the reference three-dimensional shape model on the basis of the difference between the object image and the reference object image produced on the basis of the reference three-dimensional shape model, the recovery of the three-dimensional shape model regarding the object can be effected and therefore, any special device such as slit light projecting means or means for imparting a marker and any complicated process such as an interpolating process are unnecessary.

In the apparatus and method of the present invention wherein the recovery of a three-dimensional shape model is effected on the basis of the difference between a plurality of object images from different viewing points and a plurality of reference object images corresponding thereto, the correction of the reference three-dimensional shape model is effected on the basis of the difference between a plurality of object images from different viewing point positions and a plurality a reference object images corresponding thereto and therefore, local three-dimensional structure can be corrected from different viewing point positions, and the correction of the reference three-dimensional shape model accurately reflecting the local structure can be effected.

In the apparatus of the present invention wherein a transformation coefficient for correcting the reference three-dimensional shape model found from the difference extracted by the difference extracting means is given by an affine transformation coefficient, the correction of the reference three-dimensional shape model by the shape model correcting means is effected by the use of a three-dimensional affine transformation coefficient of a point on the reference three-dimensional shape model corresponding to a point on the object image and therefore, as compared with an apparatus and method in which the correction is effected on the basis of the difference between the image data (for example, the luminance and hue data of each picture element) of the object image and the reference object image, the correction of the reference three-dimensional shape model can be more simplified.

In the apparatus of the present invention wherein the correction of the reference three-dimensional shape model by the shape model correcting means is repetitively effected, the correcting process for the reference three-dimensional shape model by the three-dimensional shape model correcting means is repetitively effected until the difference extracted by the difference extracting means becomes a predetermined threshold value or less and therefore, the correction of errors and the correction of shape data which cannot be completely covered by a single correcting process can be accomplished.

In the apparatus of the present invention wherein the reference object image after the correction of the reference three-dimensional shape model is effected by the shape model correcting means, the texture of reference object image after the correction of the reference three-dimensional shape model is corrected by the texture correcting means and therefore, more highly accurate three-dimensional shape data of the object can be obtained.

In the apparatus and method of the present invention, wherein the correction of the reference three-dimensional shape model is effected on the basis of the difference between the contour image regarding the object image and the contour image regarding the reference object image, the contour image of the reference object image corresponding to the object image is extracted on the basis of the reference three-dimensional shape model and the setting of the initial contour is effected so that the position and size of the contour image regarding the extracted reference object image may subsume the image area of the object image memorized in the image memorizing means. By this setting of the initial contour, the viewing point position and image size of the contour image regarding the reference object image become substantially the same as those of the object image memorized in the image memorizing means. Thus, in the present invention, the contour image regarding the reference object image of which the initial contour has been set becomes substantially the same as the object image in the viewing point position and the image size and therefore, by using a active contour method which will be described later with respect to some embodiments of the invention, the contour images regarding the object can be found rapidly and simply. Since as described above, the difference between these contour images also reflects the difference between the three-dimensional shapes of the object and the reference three-dimensional shape model and the difference shape model and the difference between the two-dimensional shapes or dispositions of the various portions, the reference three-dimensional shape model is corrected on the basis of that difference, whereby the recovery of the three-dimensional shape model regarding the object can be accomplished.

In the apparatus of the present invention wherein the contour image regarding the reference object image of which the difference extracted by the difference extracting means becomes minimum is extracted, the contour image regarding the reference object image set by the initial contour setting means and the contour image regarding the object image produced by the active contour processing means are image data highest in the degree of similarity and therefore, more accurate correction of the reference three-dimensional shape model can be accomplished.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 diagrammatically shows the construction of an image capturing and processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a flow chart showing the procedure of processing effected by different portion extracting means 5.

FIG. 3 is a flow chart showing the processing by different portion extracting means in an image capturing and processing apparatus according to a second embodiment of the present invention.

FIG. 4 diagrammatically shows the construction of an image capturing processing apparatus according to a third embodiment of the present invention.

FIG. 5 diagrammatically shows the construction of an image capturing and processing apparatus according to a fourth embodiment of the present invention.

FIG. 6 is a flow chart showing a correcting process effected in the image capturing and processing apparatus shown in FIG. 5, in a simplified manner.

FIG. 7 diagrammatically shows the construction of an image capturing and processing apparatus according to a fifth embodiment of the present invention.

FIG. 8 is a model view showing the manner in which an object is image captured from different viewing point positions by image pickup means 1.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Some embodiments of the present invention will hereinafter be described with reference to the drawings.

<First Embodiment>

FIG. 1 diagrammatically shows the construction of an image capturing and processing apparatus according to a first embodiment of the present invention. In FIG. 1, the reference numeral 1 designates image pickup means, the reference numeral 2 denotes reference three-dimensional shape model memorizing means, the reference numeral 3 designates first image memorizing means, the reference numeral 4 denotes second image memorizing means, the reference numeral 5 designates different portion extracting means, and the reference numeral 6 denotes shape model correcting means. These constituent portions will hereinafter be described in detail.

The image pickup means 1 is an image capturing camera provided with a zoom mechanism or the like. An object image captured by this image pickup means 1 is temporarily retained in the first image memorizing means 3.

The reference three-dimensional shape model memorizing means 2 memorizes therein in advance the reference three-dimensional shape model of a reference object which is an object, for example, a so-called wire frame model obtained by dividing the surface of the reference object by the mesh of a predetermined shape (such as a triangle), and maps gray level information or the like (texture) in each divided area onto the memorized wire frame model as it is seen from a predetermined direction to thereby produce a two-dimensional model reference object image (hereinafter referred to as the reference object image) when the reference object is seen from a predetermined direction. The reference object image produced by this reference three-dimensional shape model memorizing means 2 is memorized in the second image memorizing means 4.

The different portion extracting means 5 is means for extracting the different portion between the reference object image memorized in the second image memorizing means 4 and the object image memorized in the first image memorizing means 3. In this different portion extracting means 5, the reference object image and the object image are compared with each other and information reflecting a difference regarding three-dimensional shape information (the details of which will be described later) is extracted.

The shape model correcting means 6 is means for correcting the reference three-dimensional shape model memorized in the above-described reference three-dimensional shape model memorizing means 2, on the basis of the information reflecting the different portion extracted by the different portion extracting means 5. In this shape model correcting means 6, the reference three-dimensional shape model is corrected so that the reference object image may be approximate to an object image obtained by image capturing the object at the same viewing point position.

The specific processing by the above-described image capturing and processing apparatus will hereinafter be described.

In the reference three-dimensional shape model memorizing means 2, a typical object of which the three-dimensional shape is to be measured, for example, when the head of a particular person (a man) is an object, "the standard three-dimensional shape model of the head of man" is memorized, for example, as a wire frame model. Here, the reference three-dimensional shape model memorizing means 2 may apply mapping to "the standard three-dimensional shape model of the head head of man" memorized as the wire frame model as it is seen from a predetermined direction, to thereby produce an image as it is seen from a predetermined viewing point position under a predetermined illuminating condition or produce the image of only the wire frame as it is seen from a predetermined viewing point position. Alternatively, the viewing point position may not be preset, but although not shown, design may be made such that the output of the different portion extracting means 5 is fed back to the reference three-dimensional shape model memorizing means 2 to thereby find an optimum viewing point position and an image is produced from the found optimum viewing point position. The wire frame model to be memorized can be set in conformity with the object, and the information therefor is obtained, for example, from a memory provided internally or externally.

The reference three-dimensional shape model memorizing means 2 causes the image produced by one of the above-described methods as it is seen from a predetermined viewing point position to be memorized as the reference object image in the second image memorizing means 4, and the determination of the predetermined position may be designed to be inputted by a photographer or may be designed to be judged from the image pickup data by the image means 1. As this judging criterion, a characteristic point can be used, and a viewing point position at which the characteristic point is nearest may be adopted.

The image pickup means 1 image-captures an object of which the three-dimensional shape is to be measured from a predetermined viewing point position and causes the obtained object image to be memorized in the second image memorizing means 4. FIG. 8 is a model view showing the manner in which the object is image-captured from different viewing point positions by the image pickup means 1. In FIG. 8, the image capturing positions of the image pickup means 1 are indicated at four locations in XY plane for the sake of simplicity, but the actual image capturing position at one time is the location of the image pickup means 1 indicated, for example, by a solid line. The image capturing positions are shown only in XY plane for the sake of convenience, but if required, they are not restricted to ones in the XY plane, but may be viewing point positions in different planes or in the different visual axis direction.

In the present embodiment, the reference object image memorized in the second image memorizing means 4 and the object image memorized in the first image memorizing means 3 are adjusted so that their image sizes may substantially coincide with each other. For the adjustment of these image sizes, there are a case where the image size of the object image captured by the image pickup means 1 is adjusted to the reference object image memorized in the second image memorizing means 4 and a case where conversely, the image size of the reference object image produced by the reference three-dimensional shape model memorizing means 2 is adjusted to the object image memorized in the first image memorizing means 3.

In the case where the image size of the object image captured by the image pickup means 1 is adjusted to the reference object image, the image capturing position of the image pickup means relative to the object is adjusted so that the size of the image captured by the image pickup means 1 (the image memorized in the first image memorizing means 3) may become substantially equal to the size of the reference object image memorized in the second image memorizing means 4. This adjustment of the image capturing position of the image pickup means 1 to the object can also be effected by the use of the zooming function of the image pickup means 1.

On the other hand, in the case where the image size of the reference object image produced by the reference three-dimensional shape model memorizing means 2 is adjusted to the object image, when in the reference three-dimensional shape model memorizing means 2, for example, the reference object image standard at a predetermined viewing point position and under a predetermined illuminating condition is produced by the use of the technique of 3D computer graphics, the size of the produced reference object image is adjusted so as to become substantially equal to the size of the object image memorized in the first image memorizing means 3.

By the above-described adjustment of the image sizes of the reference object image and the object image, the coordinates system of the image captured by the image pickup means 1 (the object image memorized in the first image memorizing means 3) becomes the right-hand orthogonal coordinate system substantially coinciding with the reference coordinates system in the reference object image memorized in the second image memorizing means 4. It is to be understood that in this coordinates system, the Z-axis (not shown) is orthogonal to XY plane.

When in the manner described above, the object image and the reference object image which are substantially equal in the image size to each other are memorized in the first and second image memorizing means, respectively, the displacement of the different portion between these images, for example, the mutually corresponding points between these images, is found as a displacement vector in the following manner.

FIG. 2 is a flow chart showing the procedure effected by the different portion extracting means 5.

First, the object image (or the silhouette image thereof) and the reference object image (or the silhouette image thereof) which are substantially equal in the image size (are the same in the viewing point position or the visual axis direction) to each other are inputted from the first and second image memori