Image shake detecting device - Patent Review 6047134

Description Submit all comments and votes

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an image shake detecting device which is highly suited for the automatic image stabilizing device of a video camera or the like and also to a device for controlling the image stabilizing device.

2. Description of the Related Art

In the image sensing optical apparatuses of varied kinds including video cameras, etc., irrespective as to whether they are adapted for industrial instrumentation, or consumer appliances, shaking of an image not only hinders easy image sighting but also degrades image recognizing accuracy. In the case of the video camera, for example, the camera is often operated while the operator is walking or while the camera is on a moving vehicle. In such a case, it is inevitable to have a sensed image shaken by a shake of the camera according to the photographing conditions or the object to be photographed.

To solve this problem, there have been proposed image shake detecting devices adopting varied methods. In one of such methods, the movement of the camera is physically detected by means of an acceleration sensor (an angular velocity sensor) and an optical system is compensated for the movement according to the direction and the degree of of the movement. In another conceivable method, the parallel moving extent of of the whole image plane is detected through a video signal and is expressed in a movement vector. Then, the optical system is compensated on the basis of the vector.

In accordance with the method of using the acceleration sensor, the size of the device becomes larger thus requiring increases in space and weight. Besides, it results in a complex structural arrangement. This method is therefore hardly suited for a home video camera such as a camera-incorporating type video tape recorder which must be compact in size and light in weight.

As regards the method of computing and obtaining the movement vector of the image plane from the video signal, some camera movement that is intentionally caused by the operator might be mistakenly detected for a shake of the image. The device also would respond to a movement of the object which is in realty not a shake of the image. That method thus also has a serious problem.

The above-stated known image shake detecting devices include, for example, an image stabilizing camera which is disclosed in Japanese Laid-Open Patent Application No. SHO 61-248681. The camera of this kind is arranged as follows: an optical image is converted into an electrical signal by means of an image sensing system which consists of a lens system and a photo-electric converting element. A TV image signal is obtained through a signal processing operation performed in a given manner on the electrical signal by a signal processing circuit. The image signal thus obtained is supplied to an image shake detecting circuit as well as to a monitor. A correlation between two image planes obtained at a given time interval is detected from the image signal to find the degree and the direction of any image shake. Then, a driving control circuit and a motor are operated to control and move the lens system to offset the image shaking on the basis of the result of detection. The camera is thus arranged to be capable of obtaining a stable image even when the camera shakes.

However, the image shake detecting device which is arranged in this manner is incapable of making a discrimination between the movement of an object occurring only in a part of the image plane and a shake of the whole image plane. To solve this problem, the image shake detecting sensitivity of the device must be arranged to vary for the different areas of the image plane.

In connection with this problem, an image shake detecting device has been proposed as disclosed in an article entitled "About an Image Plane Shake Compensating Device," appeared in "The Technical Report" of the Television Society, Vol. 11, No. 3, p 43 to 48, PPOE, '87-12 (May, 1987). In the case of that device, the whole image plane is divided into 140 blocks of areas. A shake detection switch is arbitrarily turned on or off for each of these areas and an image shake is detected only from the areas for which the shake detection switch is turned on in accordance with a representing point matching method.

In accordance with the arrangement of this image shake detecting device, however, an image to be used as reference must be temporarily stored at a frame memory with its varied density values kept intact. To meet this requirement, the device necessitates the use of an analog-to-digital (hereinafter referred to as A/D) converter and a memory of a relatively large capacity. In addition to this shortcoming, the device is arranged to have one image plane superposed on another by staggering them to a certain degree of vector and to find out a vector that gives the highest degree of coincidence. Therefore, the operation of the device includes a large amount of computation. The device thus necessitates a circuit arrangement on a large scale and requires a long computing time.

Besides, it has been extremely difficult to have the image shake detecting device incorporated in a compact video camera which must be capable of carrying out real-time processing and must be arranged in a compact circuit arrangement, because: The above-stated device of the prior art necessitates the use of a large circuit arrangement including the A/D converter, the frame memory, a computing circuit, etc. and a long processing time.

Further, a control system which is provided for actually effecting shake compensation by driving a photo-taking optical system on the basis of information on a shake detected by the above-stated image shake detecting devices has been arranged as follows: Various methods have been known for correcting a displacement of an image. In one of these varied methods which has recently been developed, there are provided optical compensation members such as a lens which is arranged to have its optical axis tiltable by a motor and a variable apex-angle prism. With an image obtained through an optical system which includes these optical compensation members, the deflected degree and the size of the image to be compensated on the basis of the image displacement are detected from the image. Then, the so-called feedback control is performed to deflect the image by driving the above-stated optical compensation members in accordance with information on the results of detection.

In the case of the device of this kind, a variable apex-angle prism which is arranged to have its apex angle variable is employed as an optical image deflecting means. A motor is used for driving the prism. The deflected degree of the image is detected in the following manner: The images of at least two TV camera image planes having a time difference between them are compared with each other. The deflection of the variable apex-angle prism is then controlled in such a way as to lessen displacement occurring between the two images compared, so that the images can be obtained always in a coinciding state. Further, in cases where the object to be photographed is traced (tracked) or where the object moves, the variable apex-angle prism is driven in such a way as to lessen the degree of the positional change of the object's image within the image plane. In a case where the image is shaken by a shake of the camera caused by something, the image is stabilized by driving the variable apex-angle prism in the same way as in the case of tracing the object to lessen the degree of the positional change of the object's image. Therefore, in these cases, the control is thus performed virtually in the same manner.

However, an interlaced scanning method is used for the TV camera or the like in general. In the case of the TV camera of the NTSC system, one image plane is formed and transmitted for every period of 1/60 sec (one field period). In a case where the feedback control method is employed for detecting the state of the image obtained by such a TV camera, the data of the feedback system consists of samples dispersed at intervals of at least 1/60 sec. Besides, the feedback action delay involves a period of time required for a detection process plus a period of time required for image plane transmission (about 1/60 sec). The control output of the device, therefore, is unstable for any quick movement of the object. The response characteristic of the device is poor. In a case where the motor used for driving the optical compensation member has a good rise characteristic, a feedback coefficient exceeding 1 tends to cause oscillation. A further drawback of the prior art device resides in a poor frequency characteristic. These problems stem from the fact that the control is performed by using the control algorithm of a continuous time system while sampling by detecting means has a dispersive time delay.

Further, there has been proposed an apparatus which uses an image shake detecting device for objects appearing within an image plane and is arranged to trace (track) a moving object and to continuously perform control for accurate automatic focusing and accurate automatic exposure control. For example, apparatuses of this kind have been disclosed in U.S. patent applications Ser. No. 737,163 filed on May 23, 1985; Ser. No. 106,427 filed on Oct. 8, 1987; Ser. No. 154,078 filed on Feb. 9, 1988; Ser. No. 237,511 filed on Aug. 26, 1988; Ser. No. 240,915 filed on Sep. 6, 1988; Ser. No. 258,692 filed on Oct. 17, 1988; and Ser. No. 264,204 filed on Oct. 28, 1988. However, as mentioned in the foregoing, it is difficult to accurately make a discrimination between the movement of only one object in a part of the image plane and the movement of the whole image plane due to the movement of the camera. In accordance with the above-stated method disclosed in Japanese Laid-Open Patent Application No. SHO 61-248681, the degree of accuracy would lower if there is no luminance difference because the method utilizes a difference in luminance between the background and the object to be photographed. Further, in accordance with the technique disclosed in the above-stated Technical Report of the Television Society, it becomes difficult to accurately determine the areas in a case where a plurality of objects come to move. In other words, each of the methods of the prior art has a disadvantage as well as its advantage. Therefore, with the prior art image shake detecting devices incorporated in a compact video camera in accordance with these methods, they are incapable of adequately coping with every image condition.

With respect to the image shake detecting device of this kind, further examples have been disclosed in U.S. patent application Ser. No. 855,732 filed on Apr. 25, 1986, U.S. patent application Ser. No. 880,152 filed on Jun. 30, 1986, etc. These patent applications disclose an image stabilizing camera which is arranged to compensate the optical axis of the lens system by detecting an image shake through the edge component of the image of an object to be photographed; and an arrangement to recognize an image through a histogram which represents the distribution of the feature of the image.

SUMMARY OF THE INVENTION

This invention is directed to the solution of the above-stated problems of the prior art. It is a first object of the invention to provide an image shake detecting device which is capable of accurately and stably detecting shaking of an image plane without mistaking a movement of an image resulting from shaking of the camera for an image movement resulting from a movement of a photographing object or from a panning movement of the camera or the like.

It is another object of the invention to provide an image shake detecting device for a camera which is arranged to detect a shaking state of either the image of a main object to be photographed or that of the background, whichever suits better to the state of the object, so that shake compensation can be most appositely made under all conditions including such cases where a picture of a moving object is to be taken with the camera fixed or with the camera moved to trace (track) the moving object.

It is a further object of the invention to provide an image shake detecting device which is simple in structural arrangement and excels in responsivity without necessitating the use of any special sensor.

To attain this object, an image shake detecting device which is arranged as a preferred embodiment of this invention to detect a shake of an image on an image sensing plane on the basis of an image signal output from image sensing means comprises: first detection means for detecting the displacement of the image on the basis of an image signal corresponding to a first detection area set on the image sensing plane; second detection means for detecting the displacement of the image on the basis of an image signal corresponding to a second detection area set on the image sensing plane; switching means for selecting one of the first and second detection means; and image shake detecting means for computing the shake of the image on the basis of an output of the first or second detection means selected by the switching means.

It is a further object of the invention to provide an image shake detecting and compensating device which performs no faulty action under any image plane condition.

It is a further object of the invention to provide an image shake detecting device which performs no faulty action even in cases where a feature of a main object to be photographed and that of the background of the object are hardly distinguishable from each other due to the low level of a high frequency component or sharpness, etc. obtained within the image plane.

To attain that object, an image shake detecting device which is arranged as a preferred embodiment of the invention to detect a shake of an image on an image sensing plane on the basis of an image signal output from image sensing means comprises: movement detecting means for detecting the displacement of the image on the basis of an image signal corresponding to a predetermined detection area set on the image sensing plane; compensating means for compensating for a movement of the image, by displacing an optical axis of a photo-taking optical system which images an incident light on an imaging plane of the image sensing means, on the basis of information on the displacement of the image output from the movement detecting means; sharpness detecting means for detecting a signal component indicative of a sharpness of the image on the basis of the image signal corresponding to the detection area; and control means for controlling an action of the compensating means on the basis of an output of the sharpness detecting means.

It is a further object of the invention to provide an image shake detecting device which is arranged to measure any deviation that occurs at a given feature point within each image plane during the process of scanning image planes which differ timewise from each other and to detect a quantity of the shake of an image on the basis of a measured value thus obtained.

It is a further object of the invention to provide an image shake detecting and compensating device which has the detection sensitivity for a quantity of the shake arranged to be variable and is capable of accurately discriminating a shake of the whole image plane from a local shake occurring on the image plane, so that the device always most appositely compensates for any image shake.

It is still a further object of the invention to provide an image shake detecting device which permits processing almost in real time, as a quantity of the image shake can be briefly computed during a vertical blanking period, and also permits reduction in size, weight and cost because of a simple circuit arrangement thereof which in principle obviates the necessity of having an A/D converter and storage and computing means on a large scale.

To attain the above-stated object, an image shake detecting device which is arranged as a preferred embodiment of the invention to detect a shake of an image on an image sensing plane on the basis of an image signal output from image sensing means comprises: extracting means for extracting a feature point from within the image sensing plane; detection means for detecting a difference in generation timing of the feature point between a plurality of temporally different images; computing means for computing a quantity of the shake of the image on the basis of an output of the detection means; and sensitivity control means for varying a sensitivity of the detection means.

It is an object of the invention to provide a control device for an image sensing optical system which is very stable and has a good response characteristic.

It is another object of the invention to provide a control device for an image sensing optical system which is capable of performing an image-stabilizing and object-tracing action to effectively compensate for any image displacement resulting from a movement of an object to be photographed and is advantageously applicable not only to a photo-taking optical system but also for image processing systems of varied kinds, on account of the following arrangement thereof: In view of a sampling period for dispersively detected information and the time delay of the detected information existing in cases where the controlled system is the image sensing system of a TV camera or the like, the amount of control operation for a feedback control system is computed and supplied to the controlled system. This greatly stabilizes the control output and improves the response and frequency characteristics. With the controlled system stabilized in this manner, the setting range of feedback coefficients can be broadened for the system.

It is an object of the invention to provide a compact control device which requires no special sensor nor any special optical part.

To attain that object, a control device for an image sensing optical system which is arranged as a preferred embodiment of the invention comprises: first control means for feedback-controlling the image sensing optical system on the basis of detection information obtained by timewise dispersively detecting a state of the image sensing optical system; memory means for storing control information obtained on the basis of a quantity of control performed over the image sensing optical system when the detection information is sampled; and second control means for controlling the image sensing optical system on the basis of a result obtained by computing the control information stored in the memory means and the detection information.

Further, another control device for an image sensing optical system according to this invention comprises: scanning means for scanning at the same speed a plurality of images obtained at intervals of a predetermined period of time; detection means for detecting a time difference between the plurality of images in timing at which corresponding points of feature are generated during a scanning process of the scanning means, by counting the number of clock pulses; computing means for detecting a quantity of a shake of an image by computing an output of the detection means; and sensitivity control means for varying a detection sensitivity of the detection means by changing a period of the clock pulses.

It is an object of the invention to provide an image processing device which is capable of accurately determining an object tracing area by statistically processing movement vectors obtained at a plurality of parts of an image plane.

It is another object of the invention to provide an image processing device which determines an object tracing area so that an object desired by the photographer to be stably positioned within an image plane can be automatically discriminated from other objects and is arranged to accurately operates even in cases where there is no distinct difference in luminance between these objects or where there are a plurality of moving objects.

To attain the above-stated object, an image processing device arranged according to this invention as a preferred embodiment thereof comprises: movement vector detecting means for detecting a movement vector of an image in each of detection blocks formed by dividing an image sensing plane; computing means for detecting a distributed state of an image movement within the image sensing plane by statistically processing the movement vector obtained from each of the detection blocks; and area discriminating means for detecting an area having an image of a main photographed object on the basis of an output of the computing means.

It is an object of the invention to provide an automatic focusing device which is capable of controlling a focus adjusting device on the basis of an optical flow of a plurality of movement vectors on an image plane.

It is another object of the invention to provide an automatic focusing device which is arranged to obtain the optical flow of an image plane by detecting movement vectors obtained within a plurality of blocks set on an image sensing plane, to detect a movement of a photographed object on the basis of displacement of the optical flow, to accurately determine whether the object is approaching to or moving away from a lens and to decide the restart of an automatic focusing action on the lens according to information on the result of determination.

To attain the above-stated object, an automatic focusing device arranged as a preferred embodiment of the invention comprises: focus detecting means for detecting a degree of focusing on an object in an image sensing plane to make focus adjustment; movement vector detecting means for detecting a movement vector of an image in each of detection blocks formed by dividing the image sensing plane; area detecting means for detecting an area having an image of the object by computing the movement vectors in the detection blocks; and control means for detecting a movement quantity of the image of the object on the basis of the movement vectors in the focus detection area detected by the area detecting means and for controlling the focus detecting means on the basis of the movement quantity.

Other objects and features of the invention will become apparent from the following detailed description of embodiments thereof taken in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an image shake detecting device which is arranged according to this invention as a first embodiment thereof and is shown as in a state of being applied to a camera-incorporating type video tape recorder.

FIG. 2 is a block diagram showing the internal arrangement of the image shake detecting circuit of the first embodiment.

FIG. 3 is a block diagram showing an edge detecting circuit which is included in the image shake detecting circuit.

FIGS. 4(a) and 4(b) illustrate a shake detection area provided on an image sensing plane.

FIG. 5 is a flow chart showing the algorithm of image shake detecting and compensating actions.

FIG. 6 is a flow chart showing, as a second embodiment of the invention, the algorithm of image shake detecting and compensating actions.

FIG. 7 is a flow chart showing, as a third embodiment of the invention, the algorithm of image shake detecting and compensating actions.

FIG. 8 illustrates a method for detecting a shake occurring in an image.

FIG. 9 is a block diagram showing an image shake detecting device arranged as a fourth embodiment of the invention. The device is shown as in a state of being applied to an image stabilizing type camera.

FIGS. 10(a) to 10(c) and 11(a) to 11(c) are timing charts showing the arrangement and the operation of a gate pulse generating circuit included in the image shake detecting circuit of the invention.

FIGS. 12(a) to 12(f) show in a timing chart a method for detecting an image shake in the vertical direction.

FIGS. 13(a) and 13(b) show an arrangement for detecting an area in which a movement occurred within an image sensing plane and control over the shake detecting sensitivity.

FIG. 14 is a block diagram showing a control device of an image sensing optical system arranged according to this invention as a fifth embodiment thereof.

FIG. 15 is a block diagram showing an essential part of the fifth embodiment shown in FIG. 14.

FIG. 16 is a graph showing the tracing characteristic of the same control device.

FIG. 17 is a graph showing the result of a simulation of the tracing action of the invented control device obtained with the device actually applied to a feedback control system.

FIG. 18 is a block diagram showing a control device which is arranged according to this invention as a sixth embodiment thereof.

FIG. 19 is a block diagram showing a control device which is arranged as a seventh embodiment of the invention and is shown as in a state of being applied to an assembly robot.

FIG. 20 is a block diagram showing the arrangement of a video camera to which the invention is applied as an eighth embodiment thereof.

FIGS. 21(a) to 21(d) and 22(a) to 22(c) illustrate procedures to be taken by the eighth embodiment for determining areas.

FIG. 23 is a block diagram showing an automatic focusing device which is arranged according to this invention as a ninth embodiment thereof.

FIG. 24 is an illustration of an operation performed to determine a distance measuring area according to an optical flow.

FIG. 25 is an illustration of the nature of the optical flow.

FIG. 26 is an illustration of an operation performed to decide a restart by detecting a movement of an object within the distance measuring area.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following describes with reference to the accompanying drawings image shake detecting devices embodying this invention respectively:

FIG. 1 shows a video camera to which an image shake detecting device arranged according to this invention is applied as a first embodiment thereof. The first embodiment is characterized in that: A camera shake will never be mistaken for a movement of an object to be photographed. A panning movement of the camera is also never mistaken for a movement of the object. The device thus permits effective compensation for a camera shake.

In the case of FIG. 1, the invention is applied to a camera-incorporating type video tape recorder the arrangement of which is shown in outline in a block diagram. Referring to FIG. 1, a variable apex-angle prism 1 is formed by filling a space between two parallel flat glass plates with a liquid and by sealing the filled space with something like a rubber bellows. The apex angle of the prism 1 is thus arranged to be variable in making compensation for an image shake. The shaking movement of an optical image can be compensated for in any direction with the optical axis of the prism tilted upward, downward, to the left or to the right by applying a load to a given peripheral part of the prism 1 by means of a prism driving part 8. A photo-taking lens 11 forms a photo-taking (or image sensing) optical system in conjunction with the variable apex-angle prism 1. An image sensor 2 which is composed of a photo-electric conversion element such as CCD or the like is arranged to convert an object's image formed on the photo-electric conversion plane thereof into an electrical signal corresponding to the object's image. A camera signal processing circuit 3 consists of an amplification circuit, a matrix circuit, a gamma correction circuit, an encoder, etc. and is arranged to convert a signal received from the image sensor 2 into a luminance signal and a carrier chrominance signal. A VTR signal processing circuit 4 is arranged to frequency-modulate the luminance signal, to low-band-convert the carrier chrominance signal and then to output these processed signals in a multiplex state. A magnetic head 5 is arranged to record on a magnetic tape 6 a video signal output from the VTR signal processing circuit 4. A clock pulse generating circuit 7 is arranged to generate a clock pulse for driving the image sensor 2 and also to supply an image shake detecting circuit 9 with timing signals of varied kinds. The image shake detecting circuit 9 is arranged to detect shaking of the image and to supply an image shake compensation signal to a prism driving part 8 as will be further described later herein. An operation part 10 is provided with various operation switches including a switch for selection of an image shake compensation mode as will be further described later.

The switches arranged within the operation part 10 includes a switch SW1 which is provided for selection of ON or OFF of an image shake detecting and compensating action; and a switch SW2 which is provided for selection of one of the following modes: A mode 1 in which an image shake detection area A1 is selected; a mode 2 in which a detection area A2 is selected; a mode 3 in which the detection area A1 or A2 is automatically set; a mode 4 in which the detection areas A1 and A2 are automatically set and ON an OFF of an image shake detecting action is automatically set.

The image shake detecting device according to this invention operates to detect an image shake appositely to the state of an object's image on a principle which is as described below:

As shown in FIGS. 4(a) and 4(b), the image shake detecting device according to the invention is capable of detecting an image displacement occurring either in the detection area A1 which is within a detection frame A or in the detection area A2 which surrounds the frame A.

FIG. 4(a) shows a case in which a picture is taken with an object seized within the detection area A1 in the central part of the image sensing plane 100 and with the background fixed. In this case, the image displacement, i.e., an image shake, is detected through a video signal corresponding to the image of the fixed background in the peripheral detection area A2 by disregarding the object which is allowed to freely move in and round the central part of the image sensing plane 100.

In the case of FIG. 4(b), the camera is tracing (tracking) an object to keep it always located in the central part of the image sensing plane 100 while the object is moving at a relatively high speed. In that case, the moving object is nearly fixed within the central detection area A1 relative to the image sensing plane 100. Meanwhile, the background of the object is in a flowing state. Therefore, the shake of the image is detected through a video signal corresponding to the image obtained within the central detecting area A1. However, in cases where the image of the whole image sensing plane 100 is flowing, the image shake detecting action is not performed.

The operation part 10 enables the operator to set the camera into a desired mode by manually operating the mode selection switches SW1 and SW2. However, it is also possible to have the mode setting operation automatically performed as will be described later.

FIG. 2 shows the internal arrangement of the image shake detecting circuit 9. Referring to FIG. 2, an A/D conversion circuit 91 is arranged to convert into a digital signal the luminance signal output from the camera signal processing circuit 3. In this case, the A/D conversion does not have to be performed with a high degree of accuracy to have the luminance signal reproduced without deterioration. It may be performed merely to a patternizing degree so far as it permits detection of an image movement. In other words, the A/D conversion may be accomplished at a relatively coarse sampling frequency with a relatively coarse quantizing number of bits. An edge detecting circuit 92 is arranged to detect the edge part of the image formed on the image sensing plane 100. The circuit 92 is composed of, for example, a delay circuit 92a which has a delay time corresponding to several samples obtained by the A/D conversion circuit 91 and a subtraction circuit 92b as shown in FIG. 3. A delayed luminance signal obtained by delaying an input luminance signal is subtracted from the input luminance signal which has been converted into a digital signal. This arrangement enables the edge detecting circuit 92 to adequately detect any edge part of the object's image that suddenly changes. This edge detecting circuit 92 is provided for automatic selection of either the detection area A1 or the detection area A2 as will be further described later. The method employed for the edge detecting circuit 92 may be replaced with any other method as long as a high frequency component or the degree of sharpness of the image can be expressed by such other method. A gate circuit 93 operates under the control of a control signal coming from a control microcomputer 97. The gate circuit 93 is thus arranged to selectively pass either a video signal which corresponds to the central detection area A1 of the image sensing plane 100 or a video signal which corresponds to the peripheral detection area A2 of the image sensing plane 100. An integrating circuit 94 is arranged to integrate the video (luminance) signal passed by the gate circuit 93 for one field period. A memory 95 is arranged to be capable of storing one field period portion of the output of the A/D conversion circuit 91. A memory driving part 96 is arranged to generate a writing/reading clock pulse and an address for writing in or reading out from the memory 95 under the control of the control microcomputer 97.

Referring to FIG. 8, the image shake detecting circuit 9 which uses the video signal performs a shake detecting operation in a manner as described below:

In FIG. 8, a reference numeral 20 denotes an image sensing plane. Numerals 21 and 22 denote the locations of objects obtained at a certain point of time. A reference symbol H denotes a histogram obtained in the horizontal direction from the level of the A/D converted luminance signal which is obtained at the same point of time. A symbol V denotes a histogram obtained in the vertical direction from the level of the A/D converted luminance signal which is also obtained at the same point of time. Further, reference numerals 21' and 22' denote the locations of the objects 21 and 22 obtained after the lapse of a given period of time. The reference symbols H' and V' denote histograms obtained respectively on the basis of the luminance signal level distribution obtained then in the horizontal and vertical directions. The positions of the centers of gravity SH, SV, SH' and SV' in the horizontal and vertical directions are respectively obtained from the histograms of the different points of time. Then, movement vectors GH and GV is computed on the basis of the shift of each center of gravity taking place during the lapse of time. After that, a composite vector G is computed as follows: Vector G=GH+GV. This composite vector G represents the movement of the whole image plane, thus showing the direction and the size of a shake occurred.

The operation of the image shake detecting device embodying this invention is as described below: An incident light representing an image to be photographed is formed via the variable apex-angle prism 1 on the imaging plane of the image sensor 2. The image is photo-electric converted and output as a video signal. The video signal thus output from the image sensor 2 is processed by the camera signal processing circuit 3 and the VTR signal processing circuit 4. The processed signal is supplied to the magnetic head 5 to be recorded on the magnetic tape 6. Further, the luminance signal which is output from the camera signal processing circuit 3 is supplied to the image shake detecting circuit 9. Within the image shake detecting circuit 9 which is arranged as shown in FIG. 2, the input video signal is sampled in a given cycle by means of the A/D conversion circuit 91 to be converted into a digital signal. The digital signal thus obtained is supplied to the edge detecting circuit 92 to have an edge part of the image, i.e., a part showing a sudden level change, detected by the circuit 92. Meanwhile, the digital signal is also supplied to the memory 95 to have one field of image data stored there.

An edge signal output from the edge detecting circuit 92 is supplied to the gate circuit 93. Then, the control microcomputer 97 extracts only a portion of the edge signal representing either the detection area A1 or the detection area A2 which is selected according to the state of the image as shown in FIG. 4(a) or 4(b). The signal thus extracted is integrated for a period of one field. The integrated signal is then supplied to the control microcomputer 97. Meanwhile, data which is obtained by A/D converting one image plane portion of the luminance signal is stored by the memory 95. Then, the image information thus stored is supplied to the control microcomputer 97.

The control microcomputer 97 selects the detection area A1 or the detection area A2 by controlling the gate circuit 93. The integrated edge values of the detection areas are compared with each other to determine which of them has a larger integrated value. The luminance signal data which corresponds to the area thus determined and which is stored by the memory 95 is then read out from the memory 95 to prepare histograms for horizontal and vertical directions in a manner as described in the foregoing with reference to FIG. 8. The center of gravity of the image formed on the image sensing plane is computed and obtained from the histograms. The center of gravity thus obtained is then compared with another center of gravity which is obtained from the image on the image sensing plane after the lapse of a given period of time. By this, a movement vector G which indicates the size and the direction of image displacement, i.e., an image shake, occurred during the lapse of time is obtained by computation. Then, the data of this vector G is supplied to the prism driving part 8. In accordance with the data, the prism driving part 8 changes the apex angle of the prism 1 in such a way as to offset the size and the direction of the image displacement. A compensation thus can be effected for the image shake or displacement.

FIG. 5 shows an algorithm to be executed when the operator operates the operation part 10 of the invented image shake detecting device to manually select the shake detection area A1 or A2 (selection between the modes 1 and 2) while watching the image sensing plane. As stated in the foregoing, the operation part 10 includes, among others, the switch SW1 which is provided for causing an automatic shake detecting and compensating operation to begin or to come to a stop; and the mode selection switch SW2 which permits manual selection of either the detection area A1 or the detection area A2.

Referring to FIG. 5, the switch SW1 is first checked, at a step S1, to see if it has been turned on. If not, the flow of operation proceeds to a step S2. At the step S2, the image shake detecting circuit 9 is turned off and then the flow comes back to the step S1. If the switch SW1 is found to be on at the step S1, the flow proceeds to a step S3. Step S3: The switch SW2 is checked to see which of the central detection area A1 and the peripheral detection area A2 of the image sensing plane 100 is selected. If the peripheral detection area A2 is found to have been selected, the flow comes to a step S4. Step S4: The control microcomputer 97 reads out from the memory 95 the luminance signal data of the detection area A2 shown in FIG. 4(a) to prepare the horizontal and vertical histograms as shown in FIG. 8 and computes the centers of gravity SH2 and SV2. After that, the flow proceeds to a step S5. Step S5: The centers of gravity SH2 and SV2 are compared by computation with centers of gravity SH2" and SV2" obtained a given period before to find any difference. A vector representing any change in center of gravity is analyzed. The flow then comes to a step S6. Step S6: Image shake compensation data which is obtained by analyzing the vector is supplied to the prism driving part 8. Step S7: The variable apex-angle prism 1 is driven to adjust the optical axis thereof in the direction of offsetting the image displacement by controlling the vertical angle of the prism 1 according to the data supplied. These steps are suitable to a case where a picture is taken by seizing a main object within the central part of the image sensing plane with the background of the object fixed. An image shake can be detected in reference to the background and the prism can be compensated for the image shake.

Upon completion of the compensation, the flow of operation proceeds to a step S8. Step S8: The center-of-gravity values SH2" and SV2" which have been stored until then are replaced with the horizontal and vertical center-of-gravity values SH2 and SV2 to have the values SH2 and SV2 stored until a next detection process. The flow then comes back to the step S1.

In a case where the central detection area A1 of the image sensing plane is found to have been selected at the step S3, thus indicating that the object is moving at a relatively high speed as shown in FIG. 4(b), and that the camera is tracing (tracking) the object keeping the image of it approximately within the central detection area A1, the flow comes to a step S9 to detect an image shake in reference to the object. Step S9: The control microcomputer 97 reads out from the memory 95 the luminance signal data for the detection area A2 which is as shown in FIG. 4(b). Histograms for the horizontal and vertical directions are prepared as shown in FIG. 8. The centers of gravity SH1 and SV1 of the image are obtained from the histogram. Step S10: Then, like at the steps S5 to S7, the center-of-gravity values SH1 and SV1 are compared with previous center-of-gravity values SH1" and SV1" which are obtained, for example, for an immediately preceding field. Step S11: Information on the degree and the direction of the image shake thus obtained is supplied to the prism driving part 8. Step S12: The optical axis of the prism 1 is adjusted to compensate for the image shake. After this, the flow comes to a step S13. Step S13: The center-of-gravity values SH1 and SV1 of the current image plane are stored in place of the previous center-of-gravity values SH1" and SV1" and then the flow comes back to the step S1.

With these steps repeatedly executed, the image shake compensation can be accomplished in reference to the detection area manually selected by the operator.

While the shake detection area A1 or A2 is manually selected by the operator in the case of the flow of operation described above, the detection area can be automatically selected according to this invention. The embodiment automatically selects, according to the state of the object to be photographed, the detection area for detecting a quantity of an image shake taking place within the image sensing plane by means of the edge detecting circuit 92, the gate circuit 93 and the integrating circuit 94.

Referring to FIG. 6 which is a flow chart, the embodiment in the above-stated case operates as follows: In order to have the shake detection area A1 or A2 of the image sensing plane automatically set, the switches SW1 and SW2 of the operation part 10 are operated to select the mode 3 in which image shake detecting and detection area setting actions can be automatically performed.

At a step S21 of FIG. 6, the control microcomputer 97 which is provided within the image shake detecting circuit 9 controls and causes the gate circuit 93 to pass only the signal of the detection area A2 set outside of the detection frame A on the image sensing plane 100. As a result, edge information on the detection area A2 is supplied from the edge detecting circuit 92 to the integrating circuit 94. Then, an edge integration value obtained by the integrating action of the integrating circuit 94 is supplied to the control microcomputer 97 and is stored as data F2. The flow of operation then proceeds to a step S22. Step S22: The gate circuit 93 is controlled and caused to pass the signal of the detection area A1 which is within the detection frame A of the image sensing plane. Then an edge integration value of the detection area A1 is also supplied from the integrating circuit 94 to the microcomputer 97 as data F1. Step S23: The edge integration values of the detection areas A1 and A2 which are set inside and outside of the detection frame A are compared with each other through a data computing operation "F2-F1" to find whether the result of computation is not less than 0. In other words, a check is made to find which of the two areas has a larger image part that is effectively usable for image shake detection. In a case where the detection areas A1 and A2 differ in size, the edge integration values of them cannot be compared on an equal basis. In that case, therefore, some normalizing or weighting process is suitably performed according to the difference in size between the two detection areas.

In a case where the result of comparison between the data F2 and F1 is found at the step S23 to be F2-F