 |
Claims  |
|
|
We claim:
1. A method of processing images for constructing a target image (Io) from
adjacent images having a fixed frame line and referred to as source images
(I1, . . . , Ii, Ij, . . . , In), said source and target images having
substantially common view points, characterized in that the method
comprises the steps of:
(a) digitizing said source and target images;
(b) determining a substantially common view point (P) to said source and
target images, and a fixed orthonormal landmark (Px, Py, Pz) originated at
said common view point (P);
(c) generating, pixel by pixel, an address, for each pixel, in the target
image (Io), so as to entirely cover said target image (Io);
(d) calculating, on the basis of an address (Ao) of a current pixel
referred to as an initial pixel (m') in the target image (Io), an
orientation, in said fixed landmark, of a straight light ray (PM) passing
through said initial pixel (m') and through said common view point;
(e) selecting a source image (Ij) traversed by said straight light ray
(PM);
(f) calculating, from said orientation of the straight light ray (PM), an
address (Aq) of a distortion corrected point (m), in said selected source
image (Ij), said distortion corrected point (m) corresponding to said
initial pixel (m');
(g) calculating a luminance value (F) at said distortion correct point m in
said source image Ij;
(h) assigning the luminance value (F) of said distortion corrected point
(m) to the initial pixel (m') in the target image (Io);
(i) repeating steps (d)-(h) for each pixel of the target image (Io), until
all pixels of the target image (Io) have been processed.
2. A method as claimed in claim 1, characterized in that said method
comprises calibration steps including:
storing parameters of the source images, including scale factors (z1, . . .
, zn) and orientation angles constituted by azimuth angles (.THETA.l, . .
. , .THETA.n), angles of sight (.phi.1, . . . , .phi.n) and angles of
rotation (.psi.1, . . . , .psi.n) of source images optical axis, said
angles being defined in said fixed orthonormal landmark (Px, Py, Pz);
constructing real camera models for providing said source images with said
view point which is substantially common with the origin of said landmark,
and for correcting aberrations in said source images;
storing target image parameters, including scale factor (zo) and
orientation angles constituted by an azimuth angle (.THETA.o), an angle of
sight (.phi.o) and an angle of rotation (.psi.o) of the target image
optical axis, said angles being defined in said fixed orthonormal landmark
(Px, Py, Pz); and
constructing a virtual camera model for providing said target image with
said view point which is substantially common with that of the source
images.
3. A method as claimed in claim 2, characterized in that
the step of determining the position of the straight light ray (PM) in said
landmark comprises a geometrical transform referred to as "inverse
perspective transform" (Ho.sup.-1) which takes the virtual camera model
into account;
the step of determining the address (Aq) of said distortion corrected point
(m) corresponding to said initial pixel (m') comprises perspective
geometrical transforms (H1-Hn) referred to as "direct perspective
transforms", which, in accordance with said selected source image Ij, take
into account the respective real camera model corresponding to said
selected source image.
4. A method as claimed in claim 3, characterized in that the step of
calculating a luminance value (F) at said distortion corrected point (m) in
said selected source image (Ij) includes performing an interpolation for
computing a most probable value of a luminance function (F) at the address
(Aq) in the source image (Ij).
5. A method as claimed in claim 1, characterized in that the step of
calculating a luminance value (F) at said distortion corrected point (m) in
said selected source image (Ij) includes performing an interpolation for
computing a most probable value of a luminance function (F) at the address
(Aq) in the source image (Ij).
6. An image processing device comprising: 2
a system of n fixed real cameras (C1, . . . , Ci, Cj, . . . , Cn) arranged
in such a way that their individual fields of view merge so as to form a
single wide-angle field of view for observation of a panoramic scene, said
real cameras providing adjacent images referred to as source images;
an image construction system simulating a mobile, virtual camera (Co)
continuously scanning the panoramic scene so as to form a sub-image
referred to as target image (Io) corresponding to a selected section of
the wide-angle field of view and constructed from said source images (I1,
. . . , Ii, Ij, . . . , In) furnished by the n real cameras, characterized
in that said image processing device comprises:
means for digitizing said source and target images;
calibration means for determining a substantially common view point (P) to
said images, and a fixed orthonormal landmark (Px, Py, Pz) originated at
said common view point;
an address generator for generating, pixel by pixel, respective addresses
for the pixels of said target image (Io) so as to cover the entire target
image (Io);
an address computer for calculating, on the basis of an address (Ao) of a
current pixel referred to as initial pixel (m') in the target image (Io),
an orientation, in said fixed landmark, of a straight light ray (PM)
passing through said initial pixel (m') and through said common view point
(P) , selecting a source image (Ij) traversed by said straight light ray
(PM), calculating, from said orientation of said straight light ray (PM),
an address (Aq) of a distortion corrected point (m), in said selected
source image (Ij), said distortion corrected point (m) corresponding to
said initial point (m'); and
means for determining a luminance value (F) at said distortion corrected
point (m), and assigning said luminance value to said initial point (m') .
7. A device as claimed in claim 6, characterized in that the calibration
means further comprises:
first storage means for storing the parameters relating to said virtual
camera for supplying the address computer with a scale factor (zo) and
orientation angles of the optical axis of said virtual camera (Co) in said
fixed orthonormal landmark (Px, Py, Pz) which is independent of the
cameras, said orientation angles being constituted by an azimuth angle
(.THETA.o), an angle of sight (.phi.o) and an angle of rotation (.psi.o);
second storage means for storing parameters relating to said real cameras
(C1-Cn) for supplying said address computer with scale factor (z1-zn) and
with orientation angles of an optical axis of each real camera (C1, . . .
, Ci, Cj, Cn), said orientation angles being constituted by azimuth angles
(.THETA.1-.THETA.n), angles of sight (.phi.1-.phi.n) and angles of
rotation (.psi.1-.psi.n) defined in said fixed landmark.
8. A device as claimed in claim 7, characterized in that the address
computer comprises:
first construction means for constructing a model (MCo) of the virtual
camera with a projection via the view point P; and
second construction means for constructing models (MC1-MCn) of the real
cameras with a projection via the view point P and with corrections of
distortions.
9. A device as claimed in claim 8, characterized in that the address
computer comprises:
first means for computing a geometrical transform, referred to as "inverse
perspective transform" (H.sub.0.sup.-1), to said initial pixel (m') at an
address (Ao) of the image (Io) of the virtual camera (Co), in said inverse
perspective transform, the model of the virtual camera (MCo) provided by
said first construction means and the parameters constituted by the
azimuth angle (.THETA.o), the angle of sight (.phi.o), the angle of
rotation (.psi.o) and the scale factor (zo) of said virtual camera
provided by said first storage means being taken into account for
determining, on the basis of said inverse perspective transform
(H.sub.0.sup.-1), the positioning, in said landmark, of said straight
light ray passing through said initial pixel (m') and through the view
point (P);
means for storing the position of said straight light ray obtained by the
inverse perspective transform (H.sub.o.sup.-1);
selection means for selecting a source image (I1-In) traversed by said
straight light ray;
second means for computing a geometrical transform, referred to as "direct
perspective transform" (H1-Hn), to said straight light ray in said
landmark, said direct perspective transform, the models of the real
cameras provided by the second construction means, the parameters
constituted by the azimuth angles (.THETA.1-.THETA.n), the angles of sight
(.phi.1-.phi.n), the angles of rotation (.THETA.1-.THETA.n) and the scale
factors (z1-zn) of the respective real camera (C1-Cn) corresponding to
said selected source image provided by said second storage means being
taken into account; and
storage means for supplying, on the basis of said direct perspective
transform (H1-Hn), an address (Aq) in said selected source image (I1-In)
which corresponds to said straight light ray and thus to said initial
pixel at the address (Ao) in the target image (Io).
10. A device as claimed in claim 9, characterized in that the means for
determining the luminance comprise:
an interpolator for computing a most probable value (F) of a luminance
function at the address (Aq) found by the address computer in said
selected source image furnished by the selection means; and
third storage means for assigning said computed luminance value (F)
corresponding to the address (Aq) found in said selected source image to
the initial pixel in the target image (Io) at the address (Ao) furnished
by said address generator.
11. A device as claimed in claim 10, characterized in that said device
further comprises a display system with a screen for displaying the target
image (Io) in real time on said screen.
12. A device as claimed in claim 10, characterized in that the device
further comprises a recording system for recording the target image (Io).
13. A device as claimed in claim 10, characterized in that the system for
constructing the target image (Io) also comprises:
an interface for enabling a user to define said parameters of the virtual
camera (Co), said parameters including the scale factor (zo) and the
orientation of the optical axis (.THETA.o, .phi.o, .psi.o).
14. A device as claimed in claim 13, characterized in that the user
interface is controlled automatically or manually.
15. A device as claimed in claim 6, characterized in that the system for
constructing the target image (Io) also comprises:
an interface for enabling a user to define parameters for the virtual
camera (Co), said parameters including a scale factor (zo) and orientation
angles (.THETA.o, .phi.o, .psi.o) for the optical axis defined in said
fixed landmark of said virtual camera.
16. A device as claimed in claim 6, characterized in that said device
further comprises a display system with a screen for displaying the target
image (Io) in real time on said screen.
17. A device as claimed in claim 6, characterized in that the device
further comprises a recording system for recording the target image (Io). |
|
 |
|
 |
Claims |
|
|
|
Description |
|
|
BACKGROUND OF THE INVENTION
1. Field of the Invention
The invention relates to a method of processing images for constructing a
target image from adjacent images having a fixed frame line and referred
to as source images, said source and target images having substantially
common view points.
The invention also relates to an image processing device comprising:
a system of n fixed real cameras arranged in such a way that their
individual fields of view merge so as to form a single wide-angle field of
view for observation of a panoramic scene,
an image construction system simulating a mobile, virtual camera
continuously scanning the panoramic scene so as to form a sub-image
referred to as target image corresponding to an arbitrary section of the
wide-angle field of view and constructed from adjacent source images
furnished by the n real cameras, said virtual camera having a view point
which is common with or close to that of the real cameras.
The invention is used in the field of telemonitoring or in the field of
television where shots covering large fields are necessary, for example
when recording sports events. The invention is also used in the field of
automobile construction for realizing peripheral and panoramic rear-view
means without a blind angle.
2. Description of the Related Art
An image processing device is known from Patent Application WO 92-14341,
corresponding to U.S. Pat. No. 5,187,571. This document describes an image
processing system for television. This device comprises a transmitter
station including a plurality of fixed cameras arranged adjacent to each
other so that their fields of view merge and form a wide-angle field of
view. This system also comprises a processing station including means for
generating a composite video signal of the overall image corresponding to
the wide-angle field of view, and means for selecting a sub-image from
this composite image. This system also comprises means, such as a monitor,
for displaying this sub-image. This sub-image corresponds to a field of
view having an angle which is smaller than that of the composite image and
is referred to as sub-section of the wide-angle field of view.
This image processing device is solely suitable for conventional television
systems in which the image is formed line by line by means of a scanning
beam.
The processing station enables a user to select the sub-section of the
wide-angle field of view. The corresponding sub-image has the same
dimension as the image furnished by an individual camera. The user selects
this sub-image by varying the starting point of the scan with respect to
the composite image corresponding to the wide-angle field of view. The
wide-angle field of view has an axis which is parallel to the video scan,
with the result that the starting point for the video scan of the
sub-image may be displaced arbitrarily and continuously parallel to this
axis.
The angle of the field of view to which the sub-image corresponds may be
smaller than that of a real camera. However, the localization of the
sub-image does not include a displacement perpendicular to the scan; its
localization only includes displacements parallel to this scan. The
formation of the sub-image does not include the zoom effect with respect
to the composite image, i.e. the focal change of the sub-image with
respect to the focal length of the image pick-up cameras.
The image processing station thus comprises means for constructing the
selected video sub-image line after line. These means essentially include
a circuit for controlling the synchronization of the video signals from
the different cameras.
SUMMARY OF THE INVENTION
It is an object of the present invention to provide a device which is
capable of simulating a mobile camera scanning the wide-angle field of
view covered by the n fixed cameras whose fields of view merge.
A particular object of the present invention is to provide such a device
simulating a camera which is provided with all the facilities of a real
existing mobile camera, i.e. from a stationary observer, possibilities of
horizontal angular displacements towards the left or the right of a
panoramic scene to be observed or to be monitored, possibilites of
vertical angular displacements to the top or the bottom of this scene,
possibilities of rotation and also possibilities of zooming in on a part
of the surface area of this scene.
This object is achieved by performing a method of processing images for
constructing a target image from adjacent images having a fixed frame line
and referred to as source images, said source and target images having
substantially common view points, characterized in that the method
comprises the steps of:
digitizing the images,
determining, for one of the pixels of the target image, the address of a
corresponding point in one of all source images,
determining the luminance value at this corresponding point,
assigning the luminance value of this corresponding pixel to the initial
pixel in the target image,
repeating these steps for each pixel of the target image.
According to the invention, for performing this method, an image processing
device is also proposed, which device includes:
a system of n fixed real cameras arranged in such a way that their
individual fields of view merge so as to form a single wide-angle field of
view for observation of a panoramic scene,
an image construction system simulating a mobile, virtual camera
continuously scanning the panoramic scene so as to form a sub-image
referred to as target image corresponding to an arbitrary section of the
wide-angle field of view and constructed from adjacent source images
furnished by the n real cameras, said virtual camera having a view point
which is common with or close to that of the real cameras, characterized
in that this image processing device is a digital device and in that the
system (100) for constructing the target image Io includes:
an address computer for causing a point at an address in one of the source
images to correspond to a pixel address in the target image,
means for computing the luminance value of the point at the address found
in the source image and for assigning this luminance value to the initial
pixel at the address in the target image.
Thus, the device according to the invention provides the possibility of
constructing a target image like the one furnished by a supposed camera
which is being displaced in a continuous manner; this target image is
formed from several adjacent source images each provided by one camera
from a group of cameras arranged in a fixed manner with respect to the
scene to be observed, and, based on this construction, this device may
furnish, by way of display on the screen, or by way of recording:
either a sequential image-by-image read-out of partitions of the observed
scene, possibly with a zoom effect,
or a continuous read-out by scanning the scene observed with the sight and
azimuth effect or with rotation.
In a particular embodiment, this device is characterized in that the target
image reconstruction system comprises:
first means for storing the parameters relating to the virtual camera for
supplying the address computer with the scale factor and the orientation
of the optical axis of the virtual camera in a fixed orthonormal landmark
which is independent of the cameras, i.e. the azimuth angle, the angle of
sight and the angle of rotation;
second means for storing the parameters relating to the real cameras for
supplying the address computer with the scale factor and the orientation
of the optical axis of each real camera, i.e. their azimuth angle, the
angle of sight and the angle of rotation in said fixed landmark;
an address generator for generating, pixel by pixel, the addresses (Ao) of
the pixels of the target image so as to cover the entire target image, the
address computer determining the particular source image and the point at
the address (Aq) in this source image, which corresponds to each pixel of
the target image, on the basis of the parameters of the virtual camera and
the real cameras.
Another technical problem is posed by the construction of the target image.
It is supposed that a plurality of cameras is arranged adjacent to one
another and that no zone of the panoramic scene to be constructed is
beyond the field covered by each camera: it is thus supposed that all the
data for constructing the target image are provided. Nevertheless, at each
boundary between the cameras, where an image from one camera passes to
another image of an adjacent camera, the viewing angle difference between
these two cameras for two adjacent zones of the scene recorded by these
two different cameras causes great distortions of the image. The result is
that the partitions which are realized on and at both sides of the two
zones of the scene recorded by two different cameras are very difficult to
display and completely lack precision.
It is another object of the invention to provide a construction of the
target image whose image distortion at the boundary between two cameras is
corrected so that this (these) boundary(ies) is (are) completely invisible
to the user.
This object is achieved by means of an image processing device as described
hereinbefore, which is characterized in that the address computer
comprises:
first means for constructing a model (MCo) of the virtual camera with a
projection via the view point,
second means for constructing models (MC1-MCn) of the real cameras with a
projection via the view point and with corrections of distortions and
perspective faults.
In a particular embodiment, this device is characterized in that the
address computer comprises:
first means for computing the geometrical transform for applying a
geometrical transform referred to as inverse "perspective transform"
(H.sub.o.sup.-4) to each pixel at an address (Ao) of the image of the
virtual camera, in which transform the model (MCo) of the virtual camera
provided by the first construction means and the parameters for the
azimuth angle, the angle of sight, the angle of rotation and the scale
factor of this virtual camera provided by the first storage means are
taken into account for determining, on the basis of this inverse
perspective transform (H.sub.o.sup.-4), the positioning in said landmark
of the light ray passing through this pixel and the view point,
means for storing the position of the light ray obtained by the inverse
perspective transform (H.sub.o.sup.-4),
means for selecting the particular source image traversed by this light
ray,
second means for computing the geometrical transform for applying a
geometrical transform referred to as "direct perspective transform"
(H1-Hn) to this light ray in said landmark, in which transform the models
of the real cameras provided by the second construction means, the
parameters for the azimuth angle, the angle of sight, the angle of
rotation and the scale factor of the corresponding real camera provided by
the second storage means are taken into account,
and storage means for supplying, on the basis of this direct perspective
transform (H1-Hn), the address (Aq) in the particular source image which
corresponds to the light ray and thus to the pixel of the address (Ao) in
the target image.
With this device, the user who monitors a panoramic scene exactly obtains
the same convenience of use and the same service as a user of a mobile
camera with zoom and mechanical means for realizing the variation of the
orientation of the optical axis, i.e., for realizing variations of sight
and azimuth, as well as rotations around the optical axis of the camera.
The advantage is that the mechanical means are not necessary. These
mechanical means, which include mechanical motors for rotating the azimuth
angle and the angle of sight and a motor for zoom control always have
drawbacks: first, they may get blocked and then the generated
displacements are very slow. Moreover, they are very expensive. As they
are most frequently installed externally, they will rapidly degrade
because of poor weather conditions. The electronic image processing means
according to the invention obviate all these drawbacks because they are
very precise, reliable, very rapid and easy to control. Moreover, they may
be installed internally and thus be sheltered from bad weather. The
electronic means are also easily programmable for an automatic function.
Finally, they are less costly than the mechanical means.
With the means according to the invention, the user thus obtains an image
which is free from distortions and has a greater precision and an easier
way of carrying out the sighting operations than with mechanical means.
Moreover, a panoramic scene of a larger field may be observed because
fields of 180.degree. or even 360.degree., dependent on the number of
cameras used, can be observed. The operations can also be easily
programmed.
Great progress is achieved as regards surveillance. As for realizing
panoramic rear-view means for automobiles, this progress is also very
important.
The fact that several cameras are used for acquiring data which are
necessary for constructing the target image is not a disadvantage, because
such an assembly of fixed CCD cameras has become less difficult to handle
than the mechanical devices for varying the sight, azimuth and rotation,
as well as the zoom for a single real mobile camera.
In a particular embodiment, this system is characterized in that the means
for determining the luminance comprise:
an interpolator for computing a most probable value of a luminance function
(F) at the address (Aq) found by the address computer in the source image
furnished by the selection means;
third storage means for assigning the luminance value (F) corresponding to
the point at the address (Aq) found in the source image to the initial
pixel in the target image at the address (Ao) furnished by the address
generator, and in that the system for reconstructing the target image also
comprises:
an interface for enabling a user to define the parameters of the virtual
camera, which parameters include the scale factor and the orientation of
the optical axis.
These and other aspects of the invention will be apparent from and
elucidated with reference to the embodiments described hereinafter.
BRIEF DESCRIPTION OF THE DRAWINGS
In the drawings
FIG. 1A is a plan view showing the traces of the different image planes in
the horizontal plane of the landmark in the case where the real cameras
have image planes which are perpendicular to this horizontal plane;
FIG. 1B shows the landmark Px, Py, Pz viewed in projection in the
horizontal plane;
FIG. 1C is an elevational view of a source image plane with its particular
system of coordinate axes;
FIG. 1D is an elevational view of the target image plane with its
particular system of coordinate axes;
FIG. 1E represents the effect of limiting a section of the wide-angle field
of view of two adjacent real cameras by means of parameters chosen by the
user for the virtual camera for constructing a sub-image of a panoramic
scene;
FIG. 1F shows the target image constructed by the virtual camera defined by
these parameters, this target image being composed of a first part of an
image constructed on the basis of the source image furnished by the first
of the two real cameras and of a second image part constructed on the
basis of the source image furnished by the second of these cameras;
FIG. 1G shows an arrangement of three adjacent real cameras for covering a
field of view of 180.degree.;
FIG. 2 shows, in the form of functional blocks, the image processing device
with the system for constructing the target image, the real cameras, the
user interface and the system for displaying the target image;
FIG. 3 shows the image processing device in the form of functional blocks
in greater detail than in FIG. 2;
FIG. 4 illustrates the computation of a value of a luminance function
relative to an address in a source image;
FIG. 5A illustrates the models of the real and virtual cameras;
FIG. 5B illustrates, in projection on the horizontal plane of the landmark,
the perspective and distortion effects on the positions of the
corresponding points having the same luminance in the target image and in
the source image traversed by the same light ray passing through these
points;
FIG. 6 shows, in the form of functional blocks, the address computer which
computes the address of the point in the source image corresponding to a
pixel at an address in the target image;
FIG. 7A shows a first digital source image formed by a first real fixed
camera and FIG. 7B shows a second source image formed by a second real
fixed camera adjacent to the first camera;
FIG. 7C shows a digital target image reconstructed in the same manner as in
the case of FIG. 1F showing the distortion and perspective faults between
the first target image part constructed on the basis of the first source
image and the second target image part constructed on the basis of the
second source image; and
FIG. 7D shows the digital target image of FIG. 7C after treatment by the
image processing device, in which the distortion and perspective faults
have been eliminated.
DESCRIPTION OF THE PREFERRED EMBODIMENT
I/The image pick-up system.
FIG. 1G shows a possible arrangement of several real fixed cameras for
recording the data relating to a scene through an angle of 180.degree..
This panoramic scene is recorded with three fixed cameras C1, C2, C3. The
cameras have such optical fields that, absolutely, all the details of the
panoramic scene are recorded by the one or the other camera so that no
object under surveillance is left out. The cameras are arranged to have a
common view point P or very close view points.
The axes PZ1, PZ2, PZ3 represent the optical axes of the cameras C1, C2,
C3, respectively, and the points 01, 02, 03 represent the geometrical
centers of the images I1, I2, I3, respectively, in the image planes on the
optical axes.
A horizontal surveillance through 360.degree. can be carried out by
suitably arranging 6 fixed cameras. However, a vertical surveillance or a
surveillance in both directions may also be carried out. Those skilled in
the art will be able to realize any type of system for observation of a
panoramic scene so that a more detailed description of the various mutual
arrangements of the fixed cameras is not necessary.
With reference to FIG. 1A, the image pick-up device comprises a plurality
of n fixed real cameras having known and fixed focal lengths and being
arranged adjacent one another so that their individual fields of view
merge to cover a wide-angle field of view. The n adjacent fixed cameras
furnish n adjacent fixed images so that this image pick-up device can
monitor a panoramic scene. The cameras have such optical fields that all
the details of the panoramic scene are recorded by the one or the other
camera so that no object under surveillance is left out.
To obtain this result, these n adjacent fixed cameras are also arranged in
such a way that their optical centers P, referred to as view points
coincide. The view point of a camera is defined as the point at which each
ray emitted from a luminous source and passing through this point
traverses the optical system of the camera without any deviation.
The view points of the n cameras need not coincide physically. However, it
will hereinafter be assumed that the condition of coincidence is fulfilled
sufficiently if the distance separating each of these view points is small
as regards their distance to the filmed panoramic scene, for example, if
their respective distance is 5 cm or 10 cm and the distance to the
panoramic scene is 5 m. The condition of coincidence is thus estimated to
be fulfilled if the ratio between these distances is of the order of or is
more than 50 and, according to the invention, it is not necessary to use
costly optical mirror systems which are difficult to adjust for achieving
a strict coincidence of the view points.
II/Formation of the images by the cameras.
It is an object of the invention to provide a system for reconstructing a
digital image which simulates a mobile camera which, with the settings
selected by a user, is capable of furnishing a digital image of any part,
or sub-image, of the panoramic scene recorded by the n fixed cameras.
The n cameras are numbered C1, . . . , Ci, Cj, . . . , Cn supplying digital
source images I1, . . . , Ii, Ij, . . . , In, respectively. For example,
the source images Ii and Ij formed by two adjacent fixed real cameras Ci
and Cj will be considered hereinafter.
These fixed real cameras Ci and Cj form respective images of the panoramic
scene in adjacent source image planes Ii and Ij. In FIG. 1A the axes Pzi
and Pzj passing through the geometrical centers Oi and Oj of the source
images Ii and Ij, respectively, represent the optical axes of the fixed
real cameras Ci and Cj.
With reference to FIG. 1B, a landmark Px, Py, Pz of orthogonal axes is
defined in which the axes Px and Pz are horizontal and the axis Py is
vertical.
The source images, such as the images Ii and Ij, are numbered and each
pixel m of these images is marked by way of its coordinates in the image
plane. As is shown in FIG. 1C, a mark of rectangular coordinates (OiXi,
OiYi) and (OjXj, OjYj) are defined in each image plane in which the axes
OiXi, or OjXj are horizontal, i.e., in the plane of the landmark Px, Pz.
The image planes defined by (OiXi, OiYi) and (OjXj, OjYj) are
perpendicular to the optical axes Pzi and Pzj and have respective
geometrical centers Oi and Oj.
Once these individual marks relating to each image plane of the cameras are
established, these fixed source image planes may be related to the
landmark by means of:
their azimuth angle (or pan angle) .THETA.i, .THETA.j,
their angle of sight (or tilt angle) .phi.i, .phi.j.
The azimuth angle .THETA.i or .THETA.j is the angle forming a vertical
plane containing the optical axis PZi or PZj with the horizontal axis Pz
of the landmark. Thus, this is a horizontal angle of rotation about the
vertical axis Py.
The angle of sight .phi.i or .phi.j is the angle formed by the optical axis
PZi PZj with the horizontal plane (Px, Pz). Thus, this is a vertical angle
of rotation about a horizontal axis, the axis OiXi or OjXj of each image
plane.
For reasons of simplicity, it has hereinafter been assumed, by way of
example with reference to FIG. 1A, that the source image planes Ii, Ij
furnished by the fixed cameras Ci, Cj are vertical, i.e. their angles of
sight .phi.i, .phi.j are zero.
For similar reasons of simplicity, the same reference in FIG. 1A denotes
the trace of the planes and the axes and the corresponding planes and axes
for both the source images and for the target image described hereinafter.
FIG. 1A, which is a diagrammatic plan view of the images formed, thus only
shows the traces Ii and Ij of the fixed source image planes represented by
segments in the horizontal plane Px, Pz.
FIG. 1E shows, for example, the contiguous images Ii and Ij of the
panoramic scene, furnished by two adjacent fixed cameras Ci and Cj. In
FIG. 1E, both images Ii and Ij are projected in the same plane for the
purpose of simplicity, whereas in reality these images constitute an angle
between them which is equal to that of the optical axes of the fixed
cameras. In these images, the user may choose to observe any sub-image
bounded by the line Jo more or less to the left or to the right, more or
less to the top or to the bottom with the same magnification as the fixed
cameras or with a larger magnification, or possibly with a smaller
magnification.
The simulated mobile camera is capable of constructing a target image Io
from parts of the source image Si, Sj bounded by the line Jo in FIG. 1E.
This camera, denoted by Co hereinafter, is referred to as the virtual
camera because it simulates a camera which does not really exist.
Evidently, this simulated mobile camera is not limited to scanning the two
images Ii, Ij. It may scan all the source images from I1 to In.
This virtual camera Co can be defined in the same manner as the fixed real
camera by means of:
its azimuth angle .THETA.o
its angle of sight .phi.o
its angle of rotation .psi.o
and its magnification (zoom effect) defined by its focal length POo, and
denoted as zo, with its view point P being common with the view points P
of the fixed real cameras, while Oo is the geometrical center of the
target image Io. The view point of the virtual camera is common with the
approximate view point as defined above for the real cameras.
FIG. 1A shows the trace denoted by Io of the image plane of the virtual
camera in the horizontal plane and its optical axis PZo passing through
the geometrical centre Oo of the target image Io.
In the definition of this mobile virtual camera Co, the azimuth angle
.theta.o is the angle made by the vertical plane containing its optical
axis PZo with the horizontal axis Pz of the landmark; the angle of sight
.phi.o is the angle made by its optical axis PZo with the horizontal plane
Px, Pz of the landmark; its angle .psi.o is the angle of rotation of the
virtual camera about its own optical axis, the latter being fixed; and
finally, its focal length POo is variable so that the magnification of
t | | |
