Image processing system comprising fixed cameras and a system simulating a mobile camera

We claim:

1. An image processing system comprising:

a system of n fixed real cameras arranged in such a way that their individual fields of view merge so as to form a single wide-angle field of view for recording a panoramic scene,

an image construction system simulating a mobile, virtual camera (Cv) continuously scanning the panoramic scene to furnish a target sub-image (Iv) corresponding to an arbitrary section of the wide-angle field of view and constructed from adjacent source images (I1, . . . , Ii, Ij, . . . , In) furnished by the n real cameras,

wherein said processing system is a digital system that further comprises:

a luminance equalizing system for overall equalizing of corresponding luminance levels of first and second portions of a digital target image derived from two adjacent source images (Ii, I.sub.j), said luminance equalizing System including first and second luminance correction modules which apply a first and a second correction law (Gi, Gj), respectively, to first and second sets (R,S) of the corresponding luminance levels of the first and the second portion (Ivi, Ivj) of the digital target image derived from said two adjacent digital source images (Ii, Ij), to equalize the corresponding luminance levels to the best possible extent, in accordance with a relation Gi(R)=Gj (S).

2. An image processing system as claimed in claim 1, wherein the luminance equalizing system also includes a computation module:

which computes a law F establishing a correspondence between the first set (R) of luminance levels of the first target image portion (Ivi) and the second set (S) of luminance levels of the second target image portion (Ivj), so that S=F(R), and which thereafter computes the first and second correction laws (Gi,Gj) to be applied in the first and second luminance correction modules to equalize to the best possible extent the corresponding luminance levels in accordance with a relation Gi(R)=Gj[F(R)].

3. A system as claimed in claim 1, characterized in which further comprises, for constructing the target image (Iv) in the image construction system (GVVB):

a control module controlling the mobile virtual camera (Cv) using 3 parameters:

1) varying the panoramic orientation (PAN), designated the azimuth angle, of its optical axis, corresponding to a variation in the orientation parallel to a horizontal plane of the optical axis passing through a fixed view point common to all of the fixed real cameras and the mobile virtual camera;

2) varying the orientation (TILT), designated the angle of sight, of its optical axis, always passing through the fixed view point, in a vertical plane; and

3) varying the focal length of the virtual camera to furnish a more or less enlarged target image.

4. A system as claimed in claim 1, which further comprises modules for storing processed data of the first and second target image portions (Ivi, Ivj), and a display or recording module.

5. A system as claimed in claim 1, which further comprises:

a multiplexing module for selecting from all of the real cameras C the two cameras (Ci, Cj) which correspond to the source images (Ii, Ij) required for providing the data of the target image portions (Ivi, Ivj) used in the construction of the target image (Iv); and

two analog-to-digital converter modules for converting respective analog data supplied by the two selected real cameras (Ci, Cj) into digital data to form the two corresponding digital source images (Ii,Ij).

6. An image processing system as claimed in claim 2, wherein the computation module determines the law F for the correspondence S=F(R) as the correspondence which causes the set (r) of the luminance levels of the first pixels of each pair, in the first subset (Ai), to correspond to the best possible extent to the set(s) of the corresponding luminance levels of the second pixels of each pair in the second subset (Aj) to realise to the best possible extent the equation: s=F(r), and wherein the correspondence function F is given by F(r)=a+br.sup.c, where a, b and c are constants determined by iteration.

7. An image processing system as claimed in claim 2, wherein the computation module computes the first luminance correction law (Gi) and the second law (Gj) as linear functions of the law of correspondence F(R).

8. An image processing system as claimed in claim 7, the first correction law is chosen to be Gi(R)=kF(R)+(1-k)R, and the second correction law is chosen to be Gj(R)=k.R+(1-k)F.sup.-1 (R), wherein k is a constant which takes into account the surface proportion of one of the target image portions (Ivi, Ivj) in the overall image.

9. An image processing system as claimed in claim 2, wherein the luminance equalizing system also comprises:

a first memory module for selecting and storing a first sub-set (Ai) of pixels of the first target image portion (Ivi), comprising a first sub-set (r) of luminance levels;

a second memory module for selecting and storing a second sub-set (Aj) of pixels of the second target image portion (Ivj), comprising a second sub-set (s) of luminance levels;

and wherein the computation module computes the law F of correspondence S=F(R) by establishing the correspondence between the luminance levels of the two sub-sets (Ai, Aj) of target image portions selected for realising the equation s=F(r) to the best possible extent.

10. An image processing system as claimed in claim 9, characterized in that in the first and second memory modules, the first and second target image portions (Ivi, Ivj) furnish the first and second sub-sets (Ai, Aj):

by determining a position of a straight boundary line which, in the target image (Iv) corresponds to a border-to-border coincidence of the two target image portions (Ivi, Ivj) derived from the two adjacent source images (Ii and Ij),

by selecting, as the first and second subsets (Ai, Aj) of pixels, the pixels aligned in the rows and columns which are parallel to the straight boundary line and symmetrical with respect to said straight boundary line, at either side of the straight boundary line, in the first and second target image portions (Ivi, Ivj), respectively,

and in that in the computation module for the law of correspondence F:

the pixels of the first and second subsets (Ai, Aj), respectively, are transferred and classified by pixel pairs which are considered to correspond if they have as their coordinates the same coordinate on an axis parallel to the straight boundary line, and a coordinate which is symmetrical relative to the coordinate of the straight boundary line on an axis perpendicular to the straight boundary line, in a system of axes in the target image (Iv).

11. An image processing system as claimed in claim 9, characterized in that in the first and second memory modules, the first and second target image portions (Ivi, Ivj) are selected:

by determining an overlapping band (LiLj) which, in the target image (Iv), corresponds to an overlap of the two target image portions (Ivi, Ivj) one over the other, the limit of one band (Li) defining the parallel limit (Lj) of the other for forming the overlapping band derived from a zone in which the fields of view of the two adjacent real cameras (Ci, Cj) which furnish the source images (Ii, Ij) from which the target image portions (Ivi, Ivj) are constructed, overlap,

by selecting as the first and second subsets (Ai, Aj) of pixels, the pixels of the overlapping band located in the first and second target image portions (Ivi, Ivj), respectively,

and in that in the computation module for the law of correspondence F:

the pixels of the first and second subsets (Ai, Aj), respectively, are transferred and classified by pixel pairs which are considered to correspond if they have the same coordinates in a system of axes in the target image (Iv).

12. An image processing system as claimed in claim 10, wherein the computation module determines the law F for the correspondence S=F(R) as the correspondence which causes the set (r) of the luminance levels of the first pixels of each pair, in the first subset (Ai), to correspond to the best possible extent to the set (s) of the corresponding luminance levels of the second pixels of each pair in the second subset (Aj) to realise to the best possible extent the equation:

s=F(r).

13. An image processing system as claimed in claim 12, wherein the computation module computes the law F by means of an iterative thinning-out method by computing in a first step the function s=F(r) which approximately passes through a cluster of points N1 constituted by the luminance levels (r, s) of corresponding pairs of pixels, thereafter in a subsequent step, by weighting the points (r, s) according to their distance from a curve which is representative of the function s=F(r), the smallest weights being associated with the points most remote from said representative curve, for forming a second cluster of points N2 and by recomputing the function s=F(r) with the weighted points of said second cluster (N2), and by reducing at each step the weights of the points most remote from the curve which is representative of the function s=F(r) until the curve which is representative of the function s=F(r) passes to the best possible extent through the remaining points to which the largest weights were assigned.

14. An image processing system as claimed in claim 12, wherein the computation module computes the law F by means of an iterative thinning-out method by first computing the function s=F(r) which approximately passes through a cluster of points N1 constituted by the luminance levels (r, s) of a pair of corresponding pixels, thereafter in a subsequent step, by eliminating the points (r, s) located at a distance larger than a predetermined permissible distance from a curve which is representative of the approximative function s=F(r), to preserve a second cluster of points N1 and by recomputing the function s=F(r) using the remaining points (r, s) in the thinned-out second cluster N2, thereafter, then in each subsequent step, by reducing the value of the predetermined permissible distance between the points (r, s) and the curve which is representative of the function s=F(r), by eliminating the points (r, s) located at a distance larger than the permissible distance, and by recomputing the function s=F(r) using the remaining points (r, s), until the curve which is representative of the function s=F(r) passes to the best possible extent through the remaining points (r, s).

15. An image processing system as claimed in claim 12, wherein the correspondence function F is given by the following:

F(r)=a+br.sup.c

wherein a, b and c are constants determined by iteration, such that the computation module determines the law F by double iteration, in which each step comprises two phases in which a first phase for determining the parameters a, b, c includes a first stage in which a first value of the parameter c of the exponent is fixed and the parameters a and b are computed for a first cluster N1 of points (r, s) of subsequent stages in which the second, third . . . , etc. values of the parameter c of the exponent are fixed and the parameters a and b are computed for the same cluster N1 of points (r, s), whereafter the values of a group of parameters a, b, c are selected as furnishing the best function F(r)=a+br.sup.c passing to the best possible extent through the first cluster N1 of points (r, s), while subsequently a second thinning-out stage is carried out in which the points of the first cluster N1 most remote from the function F(r) determined in the first phase are eliminated to preserve a second cluster N2 of thinned-out points, in the subsequent step the first phase is carried out by fixing the trial values of the parameter c as the value determined in the preceding step and the values around this value, by selecting the parameters a, b, c to ensure that the function F(r)=a+br.sup.c passes to the best possible extent through the second cluster N2 of thinned-out points (r, s), and by continuing this double iteration until the function F(r)=a+br.sup.c passes to the best possible extent through the remaining points (r, s) whose distance to the curve which is representative of the function F(r) is less than a distance determined as being permissible.

16. An image processing system as claimed in claim 11, wherein the computation module determines the law F for the correspondence S=F(R) as the correspondence which causes the set (r) of the luminance levels of the first pixels of each pair, in the first subset (Ai), to correspond to the best possible extent to the set(s) of the corresponding luminance levels of the second pixels of each pair in the second subset (Aj) to realise to the best possible extent the equation:

s=F(r)

17.

17. An image processing system as claimed in claim 16, wherein the correspondence function F is given by the following:

F(r)=a+br.sup.c

wherein a, b and c are constants determined by iteration, such that the computation module determines the law F by double iteration, in which each step comprises two phases in which a first phase for determining the parameters a, b, c includes a first stage in which a first value of the parameter c of the exponent is fixed and the parameters a and b are computed for a first cluster N1 of points (r, s) of subsequent stages in which the second, third . . . , etc. values of the parameter c of the exponent are fixed and the parameters a and b are computed for the same cluster N1 of points (r, s), whereafter the values of a group of parameters a, b, c are selected as furnishing the best function F(r)=a+br.sup.c passing to the best possible extend through the first cluster N1 of points (r, s), while subsequently a second thinning-out stage is carried out in which the points of the first cluster N1 most remote from the function F(r) determined in the first phase are eliminated to preserve a second cluster N2 of thinned-out points, in the subsequent step the first phase is carried out by fixing the trial values of the parameter c as the value determined in the preceding step and the values around this value, by selecting the parameters a, b, c to ensure that the function F(r)=a+br.sup.c passes to the best possible extent through the second cluster N2 of thinned-out points (r, s), and by continuing this double iteration until the function F(r)=a+br.sup.c passes to the best possible extent through the remaining points (r s) whose distance to the curve which is representative of the function F(r) is less than a distance determined as being permissible.

18. An image processing system comprising:

a system of n fixed real cameras arranged so that their individual fields of view merge so as to form a single wide-angle field of view for recording a panoramic scene,

an image construction system simulating a mobile, virtual camera (Cv) continuously scanning the panoramic scene to construct a target sub-image (Iv) corresponding to an arbitrary section of the wide-angle field of view and derived from adjacent source images (I1, . . . , Ii, Ij . . . , In) furnished by the system of n fixed real cameras,

wherein said processing system is a digital system and further comprises:

a luminance equalizing system for overall equalizing of corresponding luminance levels of first and second portions (Ivi, Ivj) of a digital target image, where said first and second portions (Ivi, Ivj) are derived from two adjacent source images (Ii, Ij ),

said luminance equalizing system including first and second luminance correction modules for overall applying to said corresponding luminance levels of the first and second portions first and second luminance correction laws (Gi, Gj), said correction laws being evaluated, respectively, from first and second sets (R,S) of corresponding luminance levels relating to first and second sub-sets of pixels selected respectively in said first and second portions for equalizing said corresponding luminance levels of said first and second sets (R,S) to the best possible extent in accordance with the relation Gi(R)=Gj(S).

19. A system as claimed in claim 18, which further comprises, for constructing the target image (Iv) in the image construction system (GVVB):

a control module controlling the mobile virtual camera (Cv) using 3 parameters:

1) varying the panoramic orientation (PAN), designated the azimuth angle, of its optical axis, corresponding to a variation in the orientation parallel to a horizontal plane of the optical axis passing through a fixed view point common to all of the fixed real cameras and the mobile virtual camera;

2) varying the orientation (TILT), designated the angle of sight, of its optical axis, always passing through the fixed view point, in a vertical plane; and

3) varying the focal length of the virtual camera to furnish a more or less enlarged target image.

Description Submit all comments and votes

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an image processing system, comprising:

a system of n fixed real cameras arranged in such a way that their individual fields of view merge so as to form a single wide-angle field of view for recording a panoramic scene;

an image construction system simulating a mobile virtual camera continuously scanning the panoramic scene to furnish a target sub-image corresponding to an arbitrary section of the wide-angle field of view and constructed from adjacent source images furnished by the n real cameras.

This system is used in monitoring devices and in panoramic sighting devices for mobile engines.

2. Description of the Related Art

An image processing device is known from Patent Application WO 92-14341, corresponding to U.S. Pat. No. 5,187,571. This document describes an image processing system for television. This system comprises a transmitter station including a plurality of fixed cameras arranged adjacent to each other so that their fields of view merge and form a wide-angle field of view. This system also comprises a processing station including means for generating a composite video signal of the overall image corresponding to the wide-angle field of view, and means for selecting a sub-image from this composite image. This system also comprises means such as a monitor for displaying this sub-image. This sub-image corresponds to a field of view having an angle which is smaller than that of the composite image and is referred to as a sub-section of the wide-angle field of view.

This image processing system is only suitable for conventional television systems in which the image is formed line by line by means of a scanning beam.

The processing station enables a user to select the sub-section of the wide-angle field of view. The corresponding sub-image has the same dimension as the image furnished by an individual camera. The user selects this sub-image by varying the starting point of the scan with respect to the composite image corresponding to the wide-angle field of view. The wide-angle field of view has an axis which is parallel to the video scan, with the result that the starting point for the video scan of the sub-image may be displaced arbitrarily and continuously parallel to this axis.

The angle of the field of view to which the sub-image corresponds may be smaller than that of a real camera. However, the localization of the sub-image does not include a displacement perpendicular to the scan; its localization only includes displacements parallel to this scan. The formation of the sub-image does not include the zoom effect with respect to the composite image, i.e. the focal change of the sub-image with respect to the focal length of the image pick-up cameras.

The image processing station thus comprises means for constructing the selected video sub-image line after line. These means essentially include a circuit for controlling the synchronization of the video signals from the different cameras.

The present invention has for its object to solve the problem encountered when the constructed sub-image or target image is a digital image, and is computed from at least two portions of adjacent digital source images each furnished by a real fixed pick-up camera, these two cameras being arranged in such a way that their fields of view merge. In this case, a boundary line appears in the target image between the two adjacent target image portions computed from the two different source images. This is caused by the fact that each one of the two real cameras furnishes a source image which, overall, has a luminance which slightly differs from the luminance of the other source image, and that the resultant target image portions also have slightly different luminances.

This luminance difference between the two target image portions computed from the source images provided by the two adjacent cameras is the reason that the reconstructed target image is not perfect.

This boundary line is located between the picture elements computed from the picture elements of the source image of the first fixed camera, and the picture elements computed from the source image of the second fixed camera.

If the target image is constructed from the picture elements of more than two cameras, there will be as many boundary lines as there are groups of two adjacent cameras.

SUMMARY OF THE INVENTION

The present invention has for its object to provide means for equalizing the overall luminance between two adjacent digital image portions of a target image formed from digital source image portions furnished by the two real cameras whose fields of view merge.

The luminance difference between the two adjacent digital source images has several causes. Firstly, the real cameras do not respond in the same way to the same received luminous flux. This is due to the deviations in the structural features of the cameras. Furthermore, the cameras do not receive the same luminous flux since they are not identically arranged with respect to the light source of the recorded scene. Each pick-up camera is generally provided with a system for adapting its response as a function of the luminous flux received. As the luminous flux received differs from one camera to the other, the result is that their responses differ.

Moreover, even if each camera records a surface which as regards luminance is entirely uniform, the image it forms therefrom is not uniform, but causes vignetting. This is a change in the brightness of the image which generally decreases from the center towards the edges.

It appeared that two adjacent pixels, located at either side of the boundary line in the target image formed, do not originate from an area located at the same distance from the center of each original source image. Therefore, there is not the same vignetting effect in these adjacent pixels.

It may further happen that one of the pick-up cameras receives more stray light than the other--caused, for example, by an inopportune stray reflection--which also locally increases the flux received by this camera and then produces a local difference in the responses between the two cameras.

Of all these causes producing luminance differences between the two source images furnished by the two real digital cameras resulting in a difference in luminance on both sides of a boundary line in the digital image formed, it appears that said two first causes are predominant and that, basically, the annoying boundary line in the target image formed is caused by the fact that each one of the two adjacent real cameras does not overall receive the same quantity of luminous flux as the other, and/or that each one of the two cameras overall provides a different response to the same flux received.

The other causes may be disregarded, or else one can take precautions against them. For example, the stray reflections can be prevented by providing the camera with a sunshade.

It also appears that each one of the two predominant causes of the difference in luminance produces an overall effect, that is to say, the effect produced also holds uniformly for the source image from each real camera and then for the corresponding part in the target image produced.

If each one of the other causes of the luminance difference produces a local effect, vignetting, stray light zone etc., they may ultimately appear only in a small portion of the reconstructed image.

It is therefore an object of the present invention to provide means for causing the boundary line to disappear from the reconstructed target image, which fine appears between the two adjacent image portions computed from the source image picture elements supplied by the two real cameras whose fields of view merge.

According to the invention, this object is achieved by means of the image processing system defined in the opening paragraph and is characterized in that this processing system is entirely digital and that it further comprises:

a luminance equalizing system including first and second luminance correction modules (LUT.A, LUT.B), which apply a first and a second correction law (Gi, Gj), respectively, to the first and second sets (R,S) of the corresponding luminance levels of a first and a second portion (Ivi, Ivj) of a digital target image Iv formed from two adjacent digital source images (Ii, Ij), to equalize these corresponding luminance levels, in accordance with the condition Gi(R)=Gj(S).

Thus, according to the invention, a transform is made to an overall extent of, on the one hand, the different luminance levels of one of the two portions of the reconstructed target image, and, on the other hand, the different luminance levels of the other one of the two portions of the reconstructed image, employing the two laws of the overall transform of the luminance levels, so that the luminance levels of the corresponding pixels in the reconstructed target image are equalized.

Corresponding pixels are understood to mean pixels which are located at either side of the boundary line and which nevertheless correspond to two portions of the wide-angle field of view which would have the same luminance level if the image were formed by a single real camera.

At the end of such an overall treatment in accordance with a first and a second law applied to the respective luminance levels of the pixels of the first and second portions of the reconstructed image, at either side of the boundary line, it is, for example, achieved that if the first portion of the image has overall a low luminance level (dark image) and if the second portion of the image has overall a high luminance level (bright image), the luminance of the first portion is overall enhanced, and the luminance of the second portion is overall decreased, so that the corresponding pixels at either side of the boundary line have the same luminance level from that instant onwards.

This has the result that the boundary line disappears. These correction laws are applied to the pixels exclusively in dependence on their luminance level and independently of their position in each of the image portions. Nevertheless, it will be obvious that the laws to be applied must differ for either image portion to equalize the corresponding luminance levels at either side of the boundary line.

A further problem inherent in the image reconstruction for forming a target image from the elements of the two source images supplied by fixed real cameras which have fixed focal points and whose fields merge, resides in the fact that the user can cause the parameters of the device computing the target image, denoted virtual camera hereinafter, to vary. This virtual camera actually simulates a mobile camera, such as can be found in surveillance systems.

According to the invention, image processing means will be provided which are suitable for application to the image produced by a virtual camera which will be mobile and be controlled by the following 3 parameters:

1) varying the panoramic orientation (PAN), referred to as azimuth angle, of its optical axis, corresponding to a variation in the orientation parallel to a horizontal plane of this optical axis passing through a fixed view point common to both the fixed real camera and the mobile virtual camera; which panoramic variation is seen by the viewer as a control means enabling the mobile camera to be pointed to the fight or to the left;

2) varying the orientation (TILT), referred to as angle of sight, of its optical axis, always passing through the view point, in a vertical plane; a variation seen by the viewer as a control means which he can use to point the camera up or down;

3) varying the focal length of the virtual camera which renders it possible to furnish a more or less enlarged target image; a variation which the viewer sees as a ZOOM effect.

The device known from the cited document does not provide means for obtaining variants other than those of varying the azimuth and neither provides means for equalizing the luminances at either side of the boundary line in the reconstructed image of the mobile camera.

If one wants to provide the mobile virtual camera with the 3 control parameters mentioned in the foregoing, then the luminance correction means for each portion of the image at either side of a boundary line in the reconstructed target image, must be computed in real time, each time the user causes one or more of the parameters of the three control parameters of the virtual camera to vary.

Actually, it should be noted that a different contribution of each of the images of the real cameras to forming the reconstructed image corresponds to each variation of a parameter of the trio, and consequently the correction laws must be recomputed at each of these reconstructions, to take, for example, the relative importance of the different portions of the target image into account. The problem would not occur to the same extent if the reconstructed image were a fixed image or if it were formed in another manner than by digital processing.

To solve these problems, in accordance with the invention, the image processing system is further characterized in that the luminance equalizing system also includes a computation module (LUT.calcul):

which computes a law F establishing a correspondence between the levels of the first set (R) of luminance levels of the first portion of the target image (Ivi) and the levels of the second set (S) of the luminance levels of the second portion of the target image (Ivj), so that S=F(R) and which thereafter computes the first and second correction laws (Gi, Gj) to be applied in the first and second luminance correction modules (LUT.A, LUT.B) to equalize to the best possible extent the corresponding luminance levels in accordance with the relation (Gi(R)=Gj[F(R)].

Thus, the image processing system is provided with means for computing the first and second overall correction laws for the luminance according to the characteristics of the first and second portions of the image at either side of the boundary line of the reconstructed target image and for applying the first law to all the pixels of each luminance level of the first image portion and the second law to all the pixels of each luminance level of the second image portion.

As a result, the overall correction laws are computed and applied for any variation of the parameters of the mobile camera, and depending on the modifications of the target image which always appear at new parameters selected by the user.

In order to reduce the computation time, with the object of accomplishing these computations in real time, the image processing system of the invention is furthermore characterized in that the luminance equalizing system also comprises:

a first memory module (MEM.A) selecting and storing a first sub-set (Ai) of pixels of the first part of the target image (Ivi) comprising a first sub-set (r) of luminance levels;

a second memory module (MEM.B) selecting and storing a second sub-set (Aj) of pixels of the second image portion (Ivj), comprising a second sub-set (s) of luminance levels;

and in that the computation module (LUT.calcul) computes the law F of the correspondence (S=F(R) while establishing the correspondence between the luminance levels [(r) and (s)] of the two sub-sets of target image portions selected for realising the equation S=F(r) to the best possible extent.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings:

FIG. 1A is a plan view showing the traces of the different image planes in the horizontal plane of the landmark in the case where the real cameras have image planes which are perpendicular to this horizontal plane;

FIG. 1B shows the landmark Px, Py, Pz viewed in projection in the horizontal plane;

FIG. 1C is an elevational view of a real image plane with its particular system of coordinate axes;

FIG. 1D is an elevational view of the target image plane with its particular system of coordinate axes;

FIG. 2A represents the effect of limiting a section of the wide-angle field of view of two adjacent real cameras by means of parameters chosen by the user for the virtual camera for constructing a sub-image of a panoramic scene;

FIG. 2B shows the target image constructed by the virtual camera defined by these parameters, this target image being composed of a first image part constructed on the basis of the source image furnished by the first of the two real cameras and of a second image part constructed on the basis of the source image furnished by the second of these cameras when these two parts of the target images are strictly adjacent;

FIG. 2C represents the target image formed in the same manner as for the case of FIG. 2B, when the two source images supplied by the two adjacent real cameras have an area in which they overlap;

FIG. 3A represents a digital target image which has been reconstructed in the same manner as in the case of one of the FIGS. 2B or 2C, which show the total luminance difference between the first target image portion formed on the basis of the first source image and the second target image portion formed on the basis of the second source image;

FIG. 3B represents the digital target image of FIG. 3A after having been processed by the device equalizing the overall luminance;

FIG. 4A illustrates the selection of the target image portion in accordance with the border method;

FIG. 4B illustrates the selection of the target image with overlapping areas;

FIG. 5 is a graph illustrating the points of correspondence of the luminance levels (r, s) for computing the law F; and

FIG. 6 shows the circuit diagram of the image processing system in the form of functional blocks.

DESCRIPTION OF THE PREFERRED EMBODIMENT

With reference to FIG. 1A, the image pick-up device comprises a plurality of n fixed real cameras having known and fixed focal lengths and being arranged adjacent to one another so that their individual fields of view merge to cover a wide-angle field of view. The n adjacent fixed cameras furnish n adjacent fixed images so that this image pick-up device can monitor a panoramic scene. The cameras have such optical fields that all the details of the panoramic scene are recorded by the one or the other camera so that no object under surveillance is left out.

To obtain this result, these n adjacent fixed cameras are also arranged in such a way that their optical centers P, referred to as view points, coincide. The view point of a camera is defined as the point at which each ray emitted from a luminous source and passing through this point traverses the optical system of the camera without any deviation.

The view points of the n cameras need not coincide physically. However, it will hereinafter be assumed that the condition of coincidence is fulfilled sufficiently if the distance separating each of these view points is small as regards their distance to the filmed panoramic scene, for example, if their respective distance is 5 cm or 10 cm and the distance to the panoramic scene is 5 m. The condition of coincidence is thus estimated to be fulfilled if the ratio between these distances is of the order of or is more than 50 and, according to the invention, it is not necessary to use costly optical mirror systems which are difficult to adjust for achieving a strict coincidence of the view points.

The n cameras are numbered C1, . . . , Ci, Cj, . . . , Cn supplying digital source images I1, . . . , Ii, Ij, . . . , In, respectively. For example, the source images Ii and Ij formed by two adjacent fixed real cameras Ci and Cj will be considered hereinafter.

These fixed real cameras Ci and Cj form respective images of the panoramic scene in adjacent source image planes Ii and Ij. In FIG. 1A, the axes Pzi and Pzj passing through the geometrical centers Oi and Oj of the source images Ii and Ij, respectively, represent the optical axes of the fixed real cameras Ci and Cj.

With reference to FIG. 1B, a set of orthogonal axes Px, Py, Pz, positioned with respect to ground is defined in which the axes Px and Pz are horizontal and the axis Py is vertical, and is hereinafter refferd to as "landmark".

The source images, such as the images Ii and Ij, are numbered and each pixel m of these images is marked by way of its coordinates in the image plane. As is shown in FIG. 1C, a mark of rectangular coordinates (OiXi, OiYi) and (OjXj, OjYj) is defined in each image plane in which the axes OiXi, or OjXj are horizontal, i.e. in the plane of the landmark Px, Pz. The image planes defined by (OiXi, OiYi) and (OjXj, OjYj) are perpendicular to the optical axes Pzi and Pzj and have respective geometrical centers Oi and Oj.

Once these individual marks relating to each image plane of the cameras are established, these fixed real image planes may be related to the landmark by means of:

their azimuth angle (or pan angle) .theta.i, .theta.j,

their angle of sight (or tilt angle) .phi.i, .phi.j.

The azimuth angle .theta.i or .theta.j is the angle forming a vertical plane containing the optical axis PZi or PZj with the horizontal axis Pz of the landmark. Thus, this is a horizontal angle of rotation about the vertical axis Py.

The angle of sight .phi.i or .phi.j is the angle formed by the optical axis PZi or PZj with the horizontal plane (Px, Pz). Thus, this is a vertical angle of rotation about a horizontal axis, the axis OiXi or OjXj of each image plane.

For reasons of simplicity, it has hereinafter been assumed, by way of example with reference to FIG. 1A, that the real image planes Ii, Ij furnished by the fixed cameras Ci, Cj are vertical, i.e. their angles of sight .phi.i, .phi.j are zero.

For similar reasons of simplicity, the same reference in FIG. 1A denotes the trace of the planes and the axes and the corresponding planes and axes for both the source images and for the target image described hereinafter.

FIG. 1A, which is a diagrammatic plan view of the images formed, thus only shows the traces Ii and Ij of the fixed real image planes represented by segments in the horizontal plane Px, Pz.

The invention has for its object to provide a digital image reconstruction system simulating a mobile camera which is capable, by means of adjustments chosen by a user, of furnishing a digital image of any portion, or sub-image, of the panoramic scene recorded by the n fixed cameras.

FIG. 2A shows, for example, the contiguous images Ii and Ij of the panoramic scene, furnished by two adjacent fixed cameras Ci and Cj. In FIG. 2A, both images Ii and Ij are projected in the same plane for the purpose of simplicity, whereas in reality these images constitute an angle between them which is equal to that of the optical axes of the fixed cameras. In these images, the user may choose to observe any sub-image bounded by the line Io more or less to the left or to the right, more or less to the top or to the bottom with the same magnification as the fixed cameras or with a larger magnification, or possibly with a smaller magnification.

The simulated mobile camera is capable of constructing a target image Iv from parts of the source image Si, Sj bounded by the line Io in FIG. 2A. This camera is referred to as the virtual camera because it simulates a camera which does not really exist.

This virtual camera can be defined in the same manner as the fixed real cameras by means of:

its azimuth angle .theta.v

its angle of sight .phi.v

and its magnification (zoom effect) defined by its focal length POv, on condition that its view point P is common to the view points P of the fixed real cameras, while Ov is the geometrical center of the target image Iv. The view point of the virtual camera is common to the approximate view point as defined above for the real cameras.