WikiPatents - Community Patent Review
Create Free Account  |  License or Sell Your Patent  |  WikiPatents Marketplace  |  WikiPatents Blog
Username:  Password:  
    
Advanced Search
Apparatus for producing a panoramic image using a plurality of optical systems    

Get related patents on CD
United States Patent5602584   
Link to this pagehttp://www.wikipatents.com/5602584.html
Inventor(s)Mitsutake; Hideaki (Tokyo, JP); Tokumitsu; Jun (Sagamihara, JP); Suda; Shigeyuki (Yokohama, JP); Katayama; Tatsushi (Tokyo, JP)
AbstractIt is disclosed that in order to form an image of which the continuity is not lost from the image information of an area in which a double image exists and the vicinity thereof and prevent the deterioration of the image, an image processing portion to which video signals from left and right image taking systems are input has an image input portion, an image conversion portion and a synthesized image memory. The image conversion portion has a corresponding point sampling unit for sampling pairs of corresponding points corresponding to the same portion of an object from the video signals, a double image area-dividing unit for effecting the area division of a double image on the basis of the information of the three-dimensional positions of the sampled pairs of corresponding points, a double image mode-discriminating judging unit for determining an image of greater probability with which each double image is connected to a double image non-generating area, and an either image of two systems and common images of two ones simultaneous-removal unit which uses as synthesized image forming data the image determined to be more probably connected to the double image non-generating area by the double image mode-discriminating judging unit, and removes the other image.
   














 Title Information Submit all comments and votes
 
Patent Text Patent PDF Print Page Summary File History
Plain text PDF images Print Summary File History Custom Search
Inventor     Mitsutake; Hideaki (Tokyo, JP); Tokumitsu; Jun (Sagamihara, JP); Suda; Shigeyuki (Yokohama, JP); Katayama; Tatsushi (Tokyo, JP)
Owner/Assignee     Canon Kabushiki Kaisha (Tokyo, JP)
Patent assignment
All assignments
Company News
Publication Date     February 11, 1997
Application Number     08/534,744
PAIR File History     Application Data   Transaction History
Image File Wrapper   Patent Term   Fees
Litigation
Filing Date     September 27, 1995
US Classification     348/47 348/218.1 382/154
Int'l Classification     H04N 013/02 H04N 015/00 H04N 005/225
Examiner     Boudreau; Leo
Assistant Examiner     Shalwala; Bipin
Attorney/Law Firm     Morgan & Finnegan, L.L.P.
Address
Parent Case     This is a continuation of U.S. patent application Ser. No. 08/139,425, filed on Oct. 20, 1993, now abandoned.
Priority Data     Oct 22, 1992[JP]4-284343
USPTO Field of Search     348/25 348/26 348/27 348/25 348/26 348/27 348/42 348/43 348/46 348/47 348/48 348/50 348/139 348/207 348/218 382/154
Patent Tags     panoramic image plurality optical
   
Enter a comma (,) or semicolon (;) between multiple tag words/phrases.
Describe this patent:
 Amusing   
 Clever   
 Complex   
 Efficient   
 Historic   
 Important   
 Innovative   
 Interesting   
 Practical   
 Simple   
[no votes]
Patent WIKI

Share information and news about this patent, including information and news about the technology, inventors, company, ligation and licensing.

 References Submit all comments and votes
 
*references marked with an asterisk below are user-added references
 U.S. References
 
Add a new US reference:  
ReferenceRelevancyCommentsReferenceRelevancyComments
5140415
Choquet
348/46
Aug,1992

[0 after 0 votes]
4837616
Kasano
348/139
Jun,1989

[0 after 0 votes]
4825393
Nishiya
702/152
Apr,1989

[0 after 0 votes]
4819064
Diner
348/47
Apr,1989

[0 after 0 votes]
4797942
Burt
382/284
Jan,1989

[0 after 0 votes]
4601053
Grumet
382/106
Jul,1986

[0 after 0 votes]
4272756
Kakumoto
382/284
Jun,1981

[0 after 0 votes]
 Foreign References
 Other References
 Market Review Submit all comments and votes
   
Market Size
Estimate the gross annual revenues of the relevant market sector:
> $10B
$5B - $10B
$2B - $5B
$500M - $2B
$100M - $500M
$10M - $100M
$1M - $10M
$500K - $1M
$100K - $500K
< $100K
[No votes]
$0
 
$0   $2.5B   $5B   $7.5B   $10B

[0 market size comments]
Market Share
Estimate the percentage of the relevant market sector this invention will capture:
75% - 100%
50% - 74.99%
25% - 49.99%
10 - 24.99%
5 - 9.99%
2 - 4.99%
1 - 1.99%
< 1%
[No votes]
0.0%
 
0%   25%   50%   75%   100%

[0 market share comments]
Reasonable Royalty
What percentage of gross sales should the inventor or assignee be paid?
75% - 100%
50% - 74.99%
25% - 49.99%
10 - 24.99%
5 - 9.99%
2 - 4.99%
1 - 1.99%
< 1%
[No votes]
0.0%
 
0%   25%   50%   75%   100%

[0 reasonable royalty comments]
Public's "Guesstimation" of Royalty Value
Market SizeN/A[No votes]
xMarket ShareN/A[No votes]
xReasonable RoyaltyN/A[No votes]

N/A

[0 Guesstimation of Royalty Value Comments]
License Availablity
If you are NOT the owner or assignee, answer here:
Yes, license is available for purchase

No, license is not currently available



[No votes]
[0 license availability comments]
License Availablity
If you ARE the owner or assignee, answer here:
Yes, license is available for purchase

No, license is not currently available



[No votes]
[0 owner/assignee comments]
Competitive Advantage
Does this invention have a significant competitive advantage over similar technologies?
Yes

No



[No votes]
Most helpful competitive advantage comment
[No comments]

[0 competitive advantage comments]
Commercial Alternatives
Are there viable commercial alternatives for this invention?
Yes

No



[No votes]
Most helpful commercial alternative comment
[No comments]

[0 commercial alternatives comments]
 Technical Review Submit all comments and votes
 Claims Submit all comments and votes
 


What is claimed is:

1. A double-eye image pickup apparatus comprising:

a plurality of imaging optical systems;

a plurality of image pickup elements,

wherein one of said plurality of image pickup elements changes an objective image formed by one of said plurality of imaging optical systems to an image signal and another one of said plurality of image pickup elements changes another objective image formed by another one of said plurality of imaging optical systems to another image signal; and

an image processing means for generating a combined image by using image signals from said plurality of image pickup elements, wherein said image processing means executes the following steps of:

(a) extracting a plurality of pairs of corresponding points in said objective image and said another objective image,

(b) obtaining information as to a distance from each corresponding point of said pairs of corresponding points to a point on an object located in each image signal,

(c) separating said pairs of corresponding points into at least two regions on the basis of said information as to the distance to the point from each of said corresponding points, and

(d) regarding pairs of corresponding points in a region, using image data indicated by some of said corresponding points of said pairs of corresponding points for generation of a combined image and not using an image data indicated by other corresponding points of said pairs of corresponding points for generation of said combined image, a predetermined after-processing being applied to pixel portions of the image data which are not used for generation of said combined image.

2. A double-eye image pickup apparatus according to claim 1, wherein said combined image includes a panoramic synthesized image comprising said image signals juxtaposed and combined together.

3. A double-eye image pickup apparatus according to claim 2, wherein said image processing means, with regard to any pair of corresponding points of said pairs of corresponding points which belongs to an image connected to a double-image non-generating area image-picked up only by one image pickup element, uses as said combined image forming data the corresponding point of said pair of corresponding points which is most probably connected to said double image non-generating area.

4. A double-eye image pickup apparatus according to claim 2, wherein said image processing means, with regard to any pair of corresponding points of said pairs of corresponding points of which the distances from a double image non-generating area image-picked up only by one image pickup element differ, uses as said combined image forming data the corresponding point which is nearest to said double image non-generating area.

5. A double-eye image pickup apparatus according to claim 2, wherein said image processing means, with regard to any pair of corresponding points of said pairs of corresponding points which belongs to an image connected to a double image non-generating area image-picked up only by one image pickup element, uses the a corresponding point in said image as said combined image forming data, and with regard to a corresponding point belonging to another image connected to another double image non-generating area, moves the corresponding point in one of said images in the direction of juxtaposition of said images and uses it as said combined image forming data.

6. A double-eye image pickup apparatus according to claim 1, wherein said combined image is a highly minute image comprising said image signals combined together.

7. A double-eye image pickup apparatus according to claim 6, wherein said image processing means, with regard to a pair of corresponding points of said pairs of corresponding points in which the spacing between the corresponding points is smaller than a predetermined spacing, uses one corresponding point of said pair of corresponding points as said combined image forming data.

8. A double-eye image pickup apparatus according to claim 6, wherein said image processing means, with regard to at least some of said pairs of corresponding points which require a correction of distortion before the extraction of said corresponding points, uses one of said corresponding points as combined image forming data.

9. A double-eye image pickup apparatus according to claim 6, wherein said image processing means, with regard to at least some of said extracted pairs of corresponding points in which a degree of blur of image foreseen from the distance from an in-focus object plane of each of said image pickup elements and a numerical aperture of each of said imaging optical systems exceeds a predetermined threshold value, uses one corresponding point of said pairs of corresponding points as said combined image forming data.

10. A double-eye image pickup apparatus according to claim 6, wherein said image processing means, with regard to at least some of said pairs of corresponding points which appear at a predetermined spatial frequency or higher alternately with a pixel point in which corresponding points forming said pair are not found, after said pairs of corresponding points are sampled, uses one corresponding point of said pairs of corresponding points as said combined image forming data.

11. A double-eye image pickup apparatus comprising:

a plurality of imaging optical systems;

a plurality of image pickup elements,

wherein one of said plurality of image pickup elements changes an objective image formed by one of said plurality of imaging optical systems to an image signal and another one of said plurality of image pickup elements changes another objective image formed by another one of said plurality of imaging optical systems to another image signal;

an image processing means for generating a combined image by using the image signals from said plurality of image pickup elements, wherein said image processing means executes the following steps of:

(a) defining an image indicated by one of said image signals as a base image and an image indicated by said other image signal as a supplemental image,

(b) sampling a plurality of pairs of corresponding points in said base image and said supplemental image,

(c) obtaining predetermined information as to each pair of corresponding points, and

(d) from among said pairs of corresponding points on the basis of said predetermined information, using either image data indicated by said corresponding points at a side of said base image for generation of a combined image or image data indicated by said corresponding points at a side of said supplemental image for generation of said combined image, a predetermined after-processing being applied to pixel portions of the image data at said side of said supplemental image which are not used for generation of said combined image.

12. A double-eye image pickup apparatus according to claim 11, wherein said image pickup element outputting the image signal which provides said base image has an optical axis of its imaging optical system disposed toward said object.

13. An double-eye image pickup apparatus according to claim 11, wherein said combined image is a panoramic combined image comprising said image signals juxtaposed and combined together, and said image processing means, with regard to any pair of corresponding points of said pairs of corresponding points which belongs to an image connected to a double image non-generating area image-picked up only by one image pickup element, uses the image signal which is most probably connected to said double image non-generating area as a base image.

14. A double-eye image pickup apparatus according to claim 11, wherein said combined image is a highly minute image comprising said image signals combined together, and said image processing means, with regard to any pair of corresponding points of said pairs of corresponding points in which the spacing between the corresponding point in said supplemental image is smaller than a predetermined spacing, uses the corresponding point in said base image as said combined image forming data, and does not use the corresponding point in said supplemental image as said combined image forming data.

15. A double-eye image pickup apparatus according to claim 12, wherein said combined image is a highly minute image comprising said image signals combined together, and said image processing means, with regard to the pair of corresponding points of said pairs of corresponding points in which the spacing between the corresponding point in said base image and the corresponding point in said supplemental image is smaller than a predetermined spacing, uses the corresponding point in said base image as said combined image forming data, and does not use the corresponding point in said supplemental image as said combined image forming data.

16. A double-eye image pickup apparatus according to claim 11, wherein said combined image is a highly minute image comprising said image signals combined together, and said image processing means, with regard to at least some of said pairs of corresponding points which require a correction of distortion before the extraction of the corresponding points, uses the corresponding point in said base image as said combined image forming data, and does not use the corresponding point in said supplemental image as said combined image forming data.

17. A double-eye image pickup apparatus according to claim 12, wherein said combined image is a highly minute image comprising said image signals combined together, and said image processing means, with regard to at least some of said pairs of corresponding points which require a correction of distortion before the extraction of the corresponding points, uses the corresponding point in said base image as said combined image forming data, and does not use the corresponding point in said supplemental image as said combined image forming data.

18. A double-eye image pickup apparatus according to claim 11, wherein said combined image is a highly minute image comprising said image signals combined together, and said image processing means, with regard to at least some of said pairs of corresponding points in which a degree of blur of image foreseen from a distance from an in-focus object plane of each of said image pickup elements and a numerical aperture of each of said imaging optical systems exceeds a predetermined threshold value, uses the corresponding point in said base image as said combined image forming data, and does not use the corresponding point in said supplemental image as said combined image forming data.

19. A double-eye image pickup apparatus according to claim 12, wherein said combined image is a highly minute image comprising said image signals combined together, and said image processing means, with regard to at least some of said sampled pairs of corresponding points in which a degree of blur of image foreseen from a distance from an in-focus object plane of each of said image taking systems and a numerical aperture of each of said imaging optical systems exceeds a predetermined threshold value, uses the corresponding point in said base image as said combined image forming data, and does not use the corresponding point in said supplemental image as said combined image forming data.

20. A double-eye image pickup apparatus according to claim 11, wherein said combined image is a highly minute image comprising said image signals combined together, and said image processing means, with regard to at least some of said sampled pairs of corresponding points which appear at a predetermined spacial frequency or higher alternately with a pixel point in which corresponding points forming a pair are not found and an area in which the corresponding points forming said pair are not found, uses the corresponding point in said reference image as said combined image forming data, and does not use the corresponding points in said supplemental image as said combined image forming data.

21. An double-eye image pickup apparatus according to claim 12, wherein said combined image is a highly minute image comprising said image signals combined together, and said image processing means, with regard to at least some of said sampled pairs of corresponding points which appear at a predetermined spatial frequency or higher alternately with a pixel point in which corresponding points forming a pair are not found and an area in which the corresponding a points forming said pair are not found, uses the corresponding point in said base image as said combined image forming data, and does not use the corresponding points in said supplemental image as said combined image forming data.
 Description Submit all comments and votes
 


BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an image pickup apparatus using an image pickup element such as a CCD (charge coupled device) or an image pickup tube, and particularly to a double-eye image pickup apparatus using a plurality of image pickup elements and an imaging optical system such as a lens.

2. Related Background Art

In recent years, for the purpose of forming a wide panoramic image or forming a highly minute image, there has been proposed a double-eye image pickup apparatus which has a plurality of image taking systems each comprising an imaging optical system and an image pickup element and picks up the image of a common object to thereby form and output a synthesized image by the use of an image signal input from each of said image pickup elements.

A system for obtaining a panoramic image is such that the plurality of different images of an object area are picked up at a time by a plurality of image taking systems and the same object existing in these images is extracted and the respective images are connected together on the basis of relative position information in the images to thereby obtain a synthesized panoramic image.

On the other hand, a system for obtaining a highly minute image is such that as in the case of a panoramic image, the same object existing in each image is extracted and an interpolating process is carried out on the basis of relative position information in the image to thereby newly obtain a highly minute image. A double-eye image pickup apparatus based on this principle is such that as shown, for example, in FIG. 22 of the accompanying drawings, a left image taking system 1010.sub.L and a right image taking system 1010.sub.R are prepared and the image of an object 1101 is picked up by the left image taking system 1010.sub.L and the right image taking system 1010.sub.R. A left image I.sub.L obtained by the left image taking system 1010.sub.L and a right image I.sub.R obtained by the right image taking system 1010.sub.R are then synthesized by sampling a corresponding point by a CPU 1120 to thereby obtain an output image I.sub.OUT which is highly minute as compared with a case where the image of an object is picked up by a single image taking system.

SUMMARY OF THE INVENTION

However, the above-described double-eye image pickup apparatuses have suffered from the following problems. In the apparatus for obtaining a panoramic image, a plurality of images are connected together and therefore are not limited within the maximum angle of field of an image taking system and further, a panoramic image is re-formed from a plurality of images picked up at the same timing and therefore, there can be realized a flexible image pickup apparatus of a construction comprising a combination of a plurality of image taking systems enough to be able to cover a desired object area including a moving object, but images are picked up in juxtaposed relationship with one another except for some connected portions thereof and therefore, rather almost all image regions become objects having no corresponding point. In this case, objects corresponding to each other in the connected portion of the images sometimes appear as a so-called double image in which the objects do not coincide with each other on the formed image but exist doubly, and this has led to a problem that the continuity of the image is lost, thus resulting in the deteriorated dignity of the image. Here, the continuity of the image refers to the fact that in a gazed portion, the shape (outline or the like), density (luminance) ,etc. of an object are connected continuously and smoothly.

Also, in the apparatus for obtaining a highly minute image, highly accurate corresponding point relative position information (one pixel pitch or less) is required and the image interpolating method is also required to be highly accurate. Accordingly, when such accuracy is not sufficiently satisfied, high frequency noise components which originally ought not to exist may be superposed one upon another, thus resulting in rather the deterioration of the image. On the other hand, as regards time resolution, when it is supposed that this double-eye image pickup apparatus is used for television, it is necessary that about 30 frames of images per second be continuously introduced and therefore, the processing of these images must be rapid and is desired to be as simple as possible.

The present invention has been made in view of these problems and a first object thereof is to provide a double-eye image pickup apparatus for forming an image which does not lose continuity from the image formation of an area in which a double image exists and the vicinity of the area, thereby preventing the deterioration of the image.

A second object of the present invention is to provide a double-eye image pickup apparatus which satisfies the real time required by the image pickup apparatus and yet is free of the deterioration of at least the quality of image as compared with an original image before a synthesized image is formed.

To achieve the above objects, the double-eye image pickup apparatus of the present invention has a plurality of imaging optical systems, a plurality of image taking systems each comprising an image pickup element corresponding to each of said imaging optical systems, and an image processing portion for forming and outputting a synthesized image by the use of image signals input from said image pickup elements by image-picking up an object, and is characterized in that pairs of corresponding points corresponding to the same portion of said object are sampled from each of said image signals, and with regard to at least some pairs of corresponding points of said sampled pairs of corresponding points, at least one pair of corresponding points of said some pairs of corresponding points are used as synthesized image forming data and the other pairs of corresponding points are not used as the synthesized image forming data.

Also, said synthesized image may be a panoramic synthesized image comprising said image signals juxtaposed and combined together, and in this case, said image processing portion may be:

one which, with regard to the pairs of corresponding points of said sampled pairs of corresponding points which belong to an image connected to a double image non-generating area image-picked up only by one image taking system, uses are said synthetized image forming data the corresponding point of those pairs of corresponding points which belongs to an image most probably connected to said double image non-generating area;

one which, with regard to the pairs of corresponding points of said sampled pairs of corresponding points which differ in the distance to a double image non-generating area image-picked up only by one image taking system, uses as said synthesized image forming data the corresponding point of those pairs of corresponding points which is nearest to said double image non-generating area; or

one which, with regard to the pairs of corresponding points of said sampled pairs of corresponding points which belong to an image connected to one of respective double image non-generating areas image-picked up only by one image taking system, uses the corresponding point in said one image as said synthesized image forming data, and with regard to the corresponding point belonging to images connected to both of said double image non-generating areas, enlarges the corresponding point in one of said two or more images in the direction of juxtaposition of the images and uses it as said synthesized image forming data.

Further, said synthesized image may be a highly minute image provided by synthesizing said image signals, and in this case, said image processing portion may be:

one which, with regard to the pairs of corresponding points of said sampled pairs of corresponding points in which the spacing between the corresponding points is smaller than a predetermined spacing, uses one corresponding point in said pairs of corresponding points as said synthesized image forming data;

one which, with regard to at least some of said pairs of corresponding points which require the correction of distortion before the sampling of the corresponding points, uses one of said some corresponding points as said synthesized image forming data;

one which, with regard to at least some of said sampled pairs of corresponding points in which the degree of blur of image foreseen from the in-focus object plane of each said image taking system and the numerical aperture of each said imaging optical system exceeds a predetermined threshold value, uses one corresponding point of said some pairs of corresponding points as said synthesized image forming data; or

one which, with regard to at least some of the pairs of corresponding points which, after said pairs of corresponding points are sampled, appear at a predetermined spatial frequency or higher alternately with a pixel point in which corresponding points forming a pair are not found and an area in which said corresponding points forming said pair are not found, uses one corresponding point of said some pairs of corresponding points as said synthesized image forming data.

Also, the double-eye image pickup apparatus has a plurality of imaging optical systems, a plurality of image taking systems each comprising an image pickup element corresponding to each of said imaging optical systems, and an image processing portion for forming and outputting a synthesized image by the use of image signals input from said image pickup elements by image-picking up an object, and is characterized in that one of said image signals is used as a reference image for the formation of the synthesized image and the other image signal is used as an auxiliary image, pairs of corresponding points corresponding to the same portion of said object are sampled from said image signals, and with regard to some pairs of corresponding points of said sampled pairs of corresponding points, the corresponding point in said reference image is used as synthesized image forming data and the corresponding point in said auxiliary image is not used as synthesized image forming data.

In this case, said synthesized image is a panoramic synthesized image comprising said image signals juxtaposed and combined together, and said image processing portion may be one which, with regard to the pairs of corresponding points of said sampled pairs of corresponding points which belong to an image connected to a double image non-generating area image-picked up only by one image taking system, uses as the reference image the image signal most probably connected to said double image non-generating area.

Further, said image taking system having said image pickup element outputting the image signal which provides said reference image may has the optical axes of its imaging optical system disposed toward said object lying in the front direction of the apparatus,

said synthesized image is a highly minute image comprising said image signals combined together, and

said image processing portion may be one which, with regard to the pairs of corresponding points of said sampled pairs of corresponding points in which the spacing between the corresponding point in said reference image and the corresponding point in said auxiliary image is smaller than a predetermined spacing, uses the corresponding point in said reference image as said synthesized image forming data and does not use the corresponding point in said auxiliary image as said synthesized image forming data,

said synthesized image is a highly minute image comprising said image signals combined together, and

said image processing portion may be one which, with regard to at least some of said pairs of corresponding points which require the correction of distortion before the sampling of the corresponding points, uses the corresponding point in said reference image as said synthesized image forming data and does not use the corresponding point in said auxiliary image as said synthesized image forming data,

said synthesized image is a highly minute image comprising said image signals combined together, and

said image processing portion may be one which, with regard to at least some of said sampled pairs of corresponding points in which the degree of blur of image foreseen from the distance from the in-focus object plane of each said image taking system and the numerical aperture of each said imaging optical system exceeds a predetermined threshold value, uses the corresponding point in said reference image as said synthesized image forming data and does not use the corresponding point in said auxiliary image as said synthesized image forming data,

said synthesized image is a highly minute image comprising said image signals combined together, and

said image processing portion may be one which, with regard to at least some of said sampled pairs of corresponding points which appear at a predetermined spatial frequency or higher alternately with a pixel point in which corresponding points forming a pair are not found and an area in which said corresponding points forming said pair are not found, uses the corresponding point in said reference image as said synthesized image forming data and does not use the corresponding point in said auxiliary image as said synthesized image forming data.

The double-eye image pickup apparatus of the present invention constructed as described above has an image processing portion which samples pairs of corresponding points corresponding to the same portion of an object from image signals, and with regard to at least some of the sampled pairs of corresponding points, uses at least one corresponding point of said some pairs of corresponding points as synthesized image forming data and does not use the other corresponding point as the synthesized image forming data, whereby the deterioration of the synthesized image is prevented. More particularly, in the formation of a panoramic synthesized image, a double image is sometimes created near the connected portion of two images, but by at least one corresponding point of said pairs of corresponding points being used as the synthesized image forming data and the other corresponding point being not used as the synthesized image forming data, said double image is removed and also, good connection of the image with a double image non-generating area is kept. Also, in the formation of a highly minute image, when the corresponding points of the pairs of corresponding points exist very proximately to each other, at least one corresponding point of the pairs of corresponding points is used as synthesized image forming data and the other corresponding point is not used as synthesized image forming data, whereby any unnecessary high frequency component created in sensitive reaction to the error of the relative pixel position of the pairs of corresponding points or a noise component superposed on a pixel value is suppressed. Further, in the formation of a highly minute image, the processing in a distorted image portion, a blurred image portion and an occlusion image portion in which pairs of corresponding points are not found is changed as described above, whereby the omission of any unnecessary processing procedure and the suppression of the deterioration of image by a wrong countermeasure are accomplished.

Also, by setting a reference image for the formation of a synthesized image, an image having at least a predetermined level of image quality is always output. Particularly, an image taking system having an image pickup element outputting an image signal which provides a reference image has the optical axis of its imaging optical system disposed toward said object, whereby an image from at least the front of the object can always be output without being subjected to any special processing, and even during close-up photographing, the photographer can effect photographing without receiving any unnatural feeling of operation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically shows the construction of a first embodiment of the double-eye image pickup apparatus of the present invention.

FIG. 2 is a block diagram of an image processing unit shown in FIG. 1.

FIG. 3 is a view of the essential portions of the optical system of the double-eye image pickup apparatus shown in FIG. 1 as they are seen in a direction perpendicular to a plane formed by the optical axes of image pickup lens units.

FIGS. 4A and 4B are for illustrating the positional relation of an object when the image of the object is picked up by the double-eye image pickup apparatus shown in FIG. 1, FIG. 4A showing the position of the object when the object is seen in a direction similar to that in FIG. 3, and FIG. 4B being a view of the effective light receiving portions of right and left CCD sensors as they are seen from the image pickup lens unit side.

FIG. 5 shows left and right images when a composition in which a road sign is seen on the background of an automobile is image-picked up by the double-eye image pickup apparatus shown in FIG. 1.

FIG. 6 is for illustrating the tigonometrical survey method when the area division by a double image area-dividing unit shown in FIG. 2 is effected.

FIG. 7 is a histogram obtained by the double image area-dividing unit shown in FIG. 2 when the composition shown in FIG. 5 is image-picked up.

FIG. 8 shows a synthesized image obtained by synthesizing the left and right projected images shown in FIG. 5 by an image processing portion shown in FIG. 2.

FIG. 9 is a block diagram of an image processing portion in a second embodiment of the double-eye image pickup apparatus of the present invention.

FIG. 10 is an enlarged view of the area of the essential portions of the optical system shown in FIG. 3 which is indicated by hatching, for illustrating the image processing procedure of the second embodiment of the double-eye image pickup apparatus of the present invention.

FIG. 11 is a view of the essential portions of an optical system in a third embodiment of the double-eye image pickup apparatus of the present invention as they are seen in a direction perpendicular to a plane formed by the optical axes of image pickup lens units.

FIG. 12 is a view of the effective light receiving portions of left and right CCD sensors shown in FIG. 11 as they are seen from the image pickup lens unit side.

FIG. 13 is a view of the essential portions of an optical system in a fourth embodiment of the double-eye image pickup apparatus of the present invention as they are seen in a direction perpendicular to a plane formed by the optical axes of image pickup lens units.

FIG. 14 is a view of the effective light receiving portions of left and right CCD sensors shown in FIG. 13 as they are seen from the image pickup lens unit side.

FIG. 15 is a block diagram of an image processing portion in a fifth embodiment of the double-eye image pickup apparatus of the present invention.

FIGS. 16A and 16B are views when a composition in which a mountain is seen on the background of an automobile is image-picked up, for illustrating the image synthesizing process in a sixth embodiment of the double-eye image pickup apparatus of the present invention, FIG. 16A showing left and right images before image synthesis, and FIG. 16B showing an image after image synthesis.

FIG. 17 is a block diagram of an image processing portion in a seventh embodiment of the double-eye image pickup apparatus of the present invention.

FIG. 18 is a view of the essential portions of an optical system in the seventh embodiment of the double-eye image pickup apparatus of the present invention as they are seen in a direction perpendicular to a plane formed by the optical axes of image pickup lens units.

FIG. 19 shows the pixel positions of CCD sensors with respect to portions of left and right images obtained by the optical system shown in FIG. 18.

FIG. 20 is a view of the essential portions of an optical system in an eighth embodiment of the double-eye image pickup apparatus of the present invention as they are seen in a direction perpendicular to a plane formed by the optical axes of image pickup lens units.

FIG. 21 is a block diagram of an image processing portion in a ninth embodiment of the double-eye image pickup apparatus of the present invention.

FIG. 22 is a view for illustrating the principle of a double-eye image pickup apparatus for synthesizing two images obtained by picking up the image of a common object by two image taking systems to thereby obtain a highly minute image.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Some embodiments of the present invention will hereinafter be described with reference to the drawings.

FIG. 1 schematically shows the construction of a first embodiment of the double-eye image pickup apparatus of the present invention. The double-eye image pickup apparatus of the present embodiment obtains a panoramic synthesized image by parallel-connecting two images obtained by picking up the image of an object by two image taking systems, i.e., a right image taking system 10.sub.R and a left image taking system 10.sub.L.

The left image taking system 10.sub.L will first be described. The left image taking system 10.sub.L has an image pickup lens unit 11.sub.L as an imaging optical system incorporated in a lens barrel, not shown, a color resolving prism 12.sub.L for resolving light from an object into three primary colors mounted on the image pickup lens unit 11.sub.L, and three CCD sensors 13.sub.L (only one of which is shown) each having a rectangular effective light receiving portion as an image pickup element provided for each of the lights resolved by the color resolving prism 12.sub.L. The image pickup lens unit 11.sub.L is comprised of a plurality of lenses including a focusing lens unit 15.sub.L driven by a focus motor 14.sub.L and a magnification changing lens 17.sub.L driven by a zoom motor 16.sub.L, and the motors 14.sub.L and 16.sub.L are driven in conformity with control signals from an all-system control unit 21 and a focus zoom control unit 22, respectively, in a control unit 20 for controlling the optical system. On the other hand, the right image taking system 10.sub.R is also constructed similarly to the left image taking system 10.sub.L, and the optical axis L.sub.R of the image pickup lens and 11.sub.R of the right image taking system 10.sub.R is on the same plane as the optical axis L.sub.L of the image pickup lens unit 11.sub.L of the left image taking system 10.sub.L.

Also, the lens barrels in which the image pickup lens-units 11.sub.L and 11.sub.R are incorporated are coupled to the rotary shafts of convergence angle motors 18.sub.L and 18.sub.R, respectively, driven in conformity with a control signal from the convergence angle control unit 23 of the control unit 20. The rotary shafts of the convergence angle motors 18.sub.L and 18.sub.R extend in a direction perpendicular to a plane containing the optical axes L.sub.L and L.sub.R of the image pickup lens units 11.sub.L and 11.sub.R, respectively, and by the convergence angle motors 18.sub.L and 18.sub.R being driven, the image pickup lens units 11.sub.L and 11.sub.R are rotated with the color resolving prisms 12.sub.L, 12.sub.R and the CCD sensors 13.sub.L, 13.sub.R, and the angle formed by the optical axes L.sub.L and L.sub.R of the image pickup lens units 11.sub.L and 11.sub.R (the convergence angle) is set. In the respective image taking systems 10.sub.L and 10.sub.R, there are provided focus encoders 24.sub.L and 24.sub.R for detecting the positions of the focusing lens units 15.sub.L and 15.sub.R, respectively, zoom encoders 25.sub.L and 25.sub.R for detecting the positions of the magnification changing lens units 17.sub.L and 17.sub.R, respectively, and convergence angle encoders 26.sub.L and 26.sub.R for detecting the convergence angle. As these, use may be made, for example, of exterior type members like potentiometers, or members which detect the positions or angles thereof by the sig