 |
Description  |
|
|
TECHNICAL FIELD
This invention relates generally to an imaging device and method and, in
particular, to a medical imaging device and method.
BACKGROUND OF THE INVENTION
While invasive surgery may have many beneficial effects, it can cause
physical and psychological trauma to the patient from which recovery is
difficult. A variety of minimally invasive surgical procedures are
therefore being developed to minimize trauma to the patient. However,
these procedures often require physicians to perform delicate procedures
within a patient's body without being able to directly see the area of the
patient's body on which they are working. It has therefore become
necessary to develop imaging techniques to provide the medical
practitioner with information about the interior of the patient's body.
Additionally, a non-surgical or pre-surgical medical evaluation of a
patient frequently requires the difficult task of evaluating imaging from
several different modalities along with a physical examination. This
requires mental integration of numerous data sets from the separate
imaging modalities, which are seen only at separate times by the
physician.
A number of imaging techniques are commonly used today to gather two-,
three- and four-dimensional data. These techniques include ultrasound,
computerized X-ray tomography (CT), magnetic resonance imaging (MRI),
electric potential tomography (EPT), positron emission tomography (PET),
brain electrical activity mapping (BEAM), magnetic resonance angiography
(MRA), single photon emission computed tomography (SPECT),
magnetoelectro-encephalography (MEG), arterial contrast injection
angiography, digital subtraction angiography and fluoroscopy. Each
technique has attributes that make it more or less useful for creating
certain kinds of images, for imaging a particular part of the patient's
body, for demonstrating certain kinds of activity in those body parts and
for aiding the surgeon in certain procedures. For example, MRI can be used
to generate a three-dimensional representation of a patient's body at a
chosen location. Because of the physical nature of the MRI imaging
apparatus and the time that it takes to acquire certain kinds of images,
however, it cannot conveniently be used in real time during a surgical
procedure to show changes in the patient's body or to show the location of
surgical instruments that have been placed in the body. Ultrasound images,
on the other hand, may be generated in real time using a relatively small
probe. The image generated, however, lacks the accuracy and
three-dimensional detail provided by other imaging techniques.
Medical imaging systems that utilize multi-modality images and/or
position-indicating instruments are known in the prior art. Hunton, N.,
Computer Graphics World (October 1992, pp. 71-72) describes a system that
uses an ultrasonic position-indicating probe to reference MRI or CT images
to locations on a patient's head. Three or four markers are attached to
the patient's scalp prior to the MRI and/or CT scans. The resulting images
of the patient's skull and brain and of the markers are stored in a
computer's memory. Later, in the operating room, the surgeon calibrates a
sonic probe with respect to the markers (and, therefore, with respect to
the MRI or CT image) by touching the probe to each of the markers and
generating a sonic signal which is picked by four microphones on the
operating table. The timing of the signals received by each microphone
provides probe position information to the computer. Information regarding
probe position for each marker registers the probe with the MRI and/or CT
image in the computer's memory. The probe can thereafter be inserted into
the patient's brain. Sonic signals from the probe to the four microphones
will show how the probe has moved within the MRI image of the patient's
brain. The surgeon can use information of the probe's position to place
other medical instruments at desired locations in the patient's brain.
Since the probe is spacially located with respect to the operating table,
one requirement of this system is that the patient's head be kept in the
same position with respect to the operating table as well. Movement of the
patient's head would require a recalibration of the sonic probe with the
markers.
Kalawasky, R., "The Science of Virtual Reality and Virtual Environments,"
pp. 315-318 (Addison-Wesley 1993), describes an imaging system that uses a
position sensing articulated arm integrated with a three-dimensional image
processing system such as a CT scan device to provide three-dimensional
information about a patient's skull and brain. As in the device described
by Hunton, metallic markers are placed on the patient's scalp prior to the
CT scan. A computer develops a three-dimensional image of the patient's
skull (including the markers) by taking a series of "slices" or planar
images at progressive locations, as is common for CT imaging, then
interpolating between the slices to build the three-dimensional image.
After obtaining the three-dimensional image, the articulated arm can be
calibrated by correlating the marker locations with the spacial position
of the arm. So long as the patient's head has not moved since the CT scan,
the arm position on the exterior of the patient can be registered with the
three-dimensional CT image.
Heilbrun, M. P., "The Evolution and Integration of Microcomputers Used with
the Brown-Roberts-Wells (BRW) Image-guided Stereotactic System," (in
Kelly, P. J., et al. "Computers in Stereotactic Neurosurgery," pp. 43-55
(Blackwell Scientific Publications 1992)) briefly mentions the future
possibility of referencing (within the same image set) intracranial
structures to external landmarks such as a nose. However, he does not
describe how this would be accomplished, nor does he describe such a use
for multimodality image comparison or compositing.
Peters, T. M., et al., (in Kelly, P. J., et al. "Computers in Stereotactic
Neurosurgery," p. 196 (Blackwell Scientific Publications 1992)) describe
the use of a stereotactic frame with a system for using image analysis to
read position markers on each tomographic slice taken by MR or CT, as
indicated by the positions of cross-sections of N-shaped markers on the
stereotactic frame. While this method is useful for registering previously
acquired tomographic data, it does not help to register a surgeon's view
to that data. Furthermore, the technique cannot be used without a
stereotactic frame.
Goerss, S. J., "An Interactive Stereotactic Operating Suite," and Kall, B.
A., "Comprehensive Multimodality Surgical Planning and Interactive
Neurosurgery," (both in Kelly, P. J., et al. "Computers in Stereotactic
Neurosurgery," pp. 67-86, 209-229 (Blackwell Scientific Publications
1992)) describe the Compass.TM. system of hardware and software. The
system is capable of performing a wide variety of image processing
functions including the automatic reading of stereotactic frame fiducial
markers, three-dimensional reconstructions from two-dimensional data, and
image transformations (scaling, rotating, translating). The system
includes an "intramicroscope" through which computer-generated slices of a
three-dimensionally reconstructed tumor correlated in location and scale
to the surgical trajectory can be seen together with the intramicroscope's
magnified view of underlying tissue. Registration of the images is not
accomplished by image analysis, however. Furthermore, there is no mention
of any means by which a surgeon's instantaneous point of view is followed
by appropriate changes in the tomographic display. This method is also
dependent upon a stereotactic frame, and any movement of the patient's
head would presumably disable the method.
Suetens, P., et al. (in Kelly, P. J., et al. "Computers in Stereotactic
Neurosurgery," pp. 252-253 (Blackwell Scientific Publications 1992))
describe the use of a head mounted display with magnetic head trackers
that changes the view of a computerized image of a brain with respect to
the user's head movements. The system does not, however, provide any means
by which information acquired in real time during a surgical procedure can
be correlated with previously acquired imaging data.
Roberts, D. W., et al., "Computer Image Display During Frameless
Stereotactic Surgery," (in Kelly, P. J., et al. "Computers in Stereotactic
Neurosurgery," pp. 313-319 (Blackwell Scientific Publications 1992))
describe a system that registers pre-procedure images from CT, MRI and
angiographic sources to the actual location of the patient in an operating
room through the use of an ultrasonic rangefinder, an array of ultrasonic
microphones positioned over the patient, and a plurality of fiducial
markers attached to the patient. Ultrasonic "spark gaps" are attached to a
surgical microscope so that the position of the surgical microscope with
respect to the patient can be determined. Stored MRI, CT and/or
angiographic images corresponding to the microscope's focal plane may be
displayed.
Kelly, P. J. (in Kelly, P. J., et al. "Computers in Stereotactic
Neurosurgery," p. 352 (Blackwell Scientific Publications 1992)) speculates
about the future possibility of using magnetic head tracking devices to
cause the surgical microscope to follow the surgeon's changing field of
view by following the movement within the established three-dimensional
coordinate system. Insufficient information is given to build such a
system, however. Furthermore, this method would also be stereotactic frame
dependent, and any movement of the patient's head would disable the
coordinate correlation.
Krueger, M. W., "The Emperor's New Realities," pp. 18-33, Virtual Reality
World (Nov./Dec. 1993) describes generally a system which correlates real
time images with stored images. The correlated images, however, are of
different objects, and the user's point of view is not tracked.
Finally, Stone, R. J., "A Year in the Life of British Virtual Reality", p.
49-61, Virtual Reality World (Jan./Feb. 1994) discusses the progress of
Advanced Robotics Research Limited in developing a system for scanning
rooms with a laser rangefinder and processing the data into simple
geometric shapes "suitable for matching with a library of a priori
computer-aided design model primitives." While this method seems to
indicate that the group is working toward generally relating two sets of
images acquired by different modalities, the article provides no means by
which such matching would be accomplished. Nor does there seem to be
classification involved at any point. No means are provided for acquiring,
processing, and interacting with image sets in real time, and no means are
provided for tracking the instantaneous point of view of a user who is
performing a procedure, thereby accessing another data set.
As can be appreciated from the prior art, it would be desirable to have an
imaging system capable of displaying single modality or multimodality
imaging data, in multiple dimensions, in its proper size, rotation,
orientation, and position, registered to the instantaneous point of view
of a physician examining a patient or performing a procedure on a patient.
Furthermore, it would be desirable to do so without the expense,
discomfort, and burden of affixing a stereotactic frame to the patient in
order to accomplish these goals. It would also be desirable to utilize
such technology for non-medical procedures such as the repair of a device
contained within a sealed chassis.
SUMMARY OF THE INVENTION
This invention provides a method and apparatus for obtaining and displaying
in real time an image of an object obtained by one modality such that the
image corresponds to a line of view established by another modality. In a
preferred embodiment, the method comprises the following steps: obtaining
a follow image library of the object via a first imaging modality;
providing a lead image library obtained via the second imaging modality;
referencing the lead image library to the follow image library; obtaining
a lead image of the object in real time via the second imaging modality
along a lead view; comparing the real time lead image to lead images in
the lead image library via digital image analysis to identify a follow
image line of view corresponding to the lead view; transforming the
identified follow image to correspond to the scale, rotation and position
of the lead image; and displaying the transformed follow image, the
comparing, transforming and displaying steps being performed substantially
simultaneously with the step of obtaining the lead image in real time.
The invention is described in further detail below with reference to the
drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram showing a preferred embodiment of the imaging
device of this invention.
FIG. 2 is a flow chart illustrating a preferred embodiment of the method of
this invention.
FIG. 3 is a flow chart illustrating an alternative embodiment of the method
of this invention.
DETAILED DESCRIPTION OF THE INVENTION
Definitions
The following definitions are useful in understanding and using the device
and method of the invention.
Image. As used herein "image" means the data that represents the spacial
layout of anatomical or functional features of a patient, which may or may
not be actually represented in visible, graphical form. In other words,
image data sitting in a computer memory, as well as an image appearing on
a computer screen, will be referred to as an image or images. Non-limiting
examples of images include an MRI image, an angiography image, and the
like. When using a video camera as a data acquisition method, an "image"
refers to one particular "frame" in the series that is appropriate for
processing at that time. Because the ability to "re-slice" a
three-dimensional reconstruction of a patient's body in a plane
corresponding to the trajectory of the "lead view" (typically the line of
view from which the surgeon wishes to view the procedure) is important to
this method, the "image" may refer to an appropriately re-sliced image of
a three-dimensional image reconstruction, rather than one of the
originally acquired two-dimensional files from which the reconstructions
may have been obtained. The term image is also used to means any portion
of an image that has been selected, such as a fiducial marker, subobject,
or knowledge representation.
Imaging modality. As used herein "imaging modality" means the method or
mechanism by which an image is obtained, e.g., MRI, CT, video, ultrasound,
etc.
Lead view. As used herein "lead view" means the line of view toward the
object at any given time. Typically the lead view is the line of view
through which the physician at any given time wishes to view the
procedure. In the case where a see-through head-mounted display and
head-mounted camera are utilized, this should be the instantaneous line of
view of the physician. As the lead view shifts, all other images must
adjust their views to that of the lead view in order to make all of the
images that converge to make a resulting composite image accurate.
Lead image. As used herein "lead image" is an image obtained through the
same modality as the lead view. For example, if the lead view is the
physician's view of the surface of the patient, the lead image could be a
corresponding video image of the surface of the patient.
Follow image. As used herein "follow image" will be an image which must be
transformed and possibly sliced to the specifications of the lead view and
slice depth control. A properly sliced and transformed follow image will
usually be in a plane parallel with that of the lead image, and
consequently, orthogonal to the lead view, although other slice contours
could be used. A properly transformed follow image will be at the same
angle of the view as the lead image, but at a depth to be separately
determined.
Composite image. As used herein "composite image" is the image that results
from the combination of properly registered lead and follow images from
two or more sources, each source representing a different modality.
Fiducial marker. As used herein "fiducial marker" means a feature, image
structure, or subobject present in lead or follow images that can be used
for image analysis, matching, coordinate interreferencing or registration
of the images and creation of a composite image.
Feature extraction. As used herein "feature extraction" means a method of
identification of image components which are important to the image
analysis being conducted. These may include boundaries, angles, area,
center of mass, central moments, circularity, rectangularity and regional
gray-scale intensities in the image being analyzed.
Segmentation. As used herein "segmentation" is the method of dividing an
image into areas which have some physical significance in terms of the
original scene that the image attempts to portray. For example,
segmentation may include the demarcation of a distinct anatomical
structure, such as an external auditory meatus, although it may not be
actually identified as such until classification. Thus, feature extraction
is one method by which an image can be segmented. Additionally, previously
segmented areas may be subsequently subjected to feature extraction. Other
non-limiting examples of methods of segmentation which are well known in
the area of image analysis include: thresholding, edge detection, Hough
transform, region growing, run-length connective analysis, boundary
analysis, template matching and the like. See, e.g., Rosenfeld, A., "The
fuzzy geometry of image subsets," (in Bezdek, J. C. et al., "Fuzzy Models
for Pattern Recognition," pp. 340-346 (IEEE 1992)).
Classification. As used herein "classification" means a step in the imaging
method of the invention in which an object is identified as being of a
certain type, based on its features. For example, a certain segmented
object in an image might be identified by a computer as being an external
auditory meatus based on if it falls within predetermined criteria for
size, shape, pixel density, and location relative to other segmented
objects. In this invention, classification is extended to include the
angle, or Cartesian location, from which the object is viewed ("line of
view"), for example, an external auditory meatus viewed from 30.degree.
North and 2.degree. West of a designated origin. A wide variety of
classification techniques are known, including statistical techniques
(see, e.g., Davies, E. R., "Machine Vision: Theory, Algorithms,
Practicalities," pp. 435-451 (Academic Press 1992)) and fuzzy logic
techniques (see, e.g., Bezdek, J. C., et al., "Fuzzy Models for Pattern
Recognition," pp. 1-27 (IEEE 1992); Siy, P., et al., "Fuzzy Logic for
Handwritten Numeral Character Recognition," (in Bezdek, J. C., et al.,
"Fuzzy Models for Pattern Recognition," pp. 321-325 (IEEE 1992)).
Classification techniques are discussed in Faugeras, "Three-Dimensional
Computer-Vision," pp. 483-558 (MIT Press 1989) and Haralick, R. M., et
al., "Computer and Robot Vision," vol. 2, pp. 43-185, 289-378, 493-533
(Addison-Wesley 1993).
Transformation. As used herein, "transformation" means processing an image
such that it is translated (moved in a translational fashion), rotated (in
two or three dimensions), scaled, sheared, warped, placed in perspective
or otherwise altered according to specified criteria. See Burger, P.,
"Interactive Computer Graphics," pp. 173-186 (Addison-Wesley 1989).
Registration. As used herein, "registration" means alignment process by
which two images of like or corresponding geometries and of the same set
of objects are positioned coincident with each other so that corresponding
points of the imaged scene appear in the same position on the registered
images.
Description of the Preferred Embodiments
For convenience, the preferred embodiment of the invention is discussed in
the context of medical applications, such as in brain surgery or other
invasive surgeries. The invention is also applicable to other uses,
including but not limited to medical examinations, analysis of ancient and
often fragile artifacts, airplane luggage, chemical compositions (in the
case of nuclear magnetic resonance spectral analysis); the repair of
closed pieces of machinery through small access ways; and the like.
The invention improves earlier methods and devices for creating
multimodality composite images by providing a new way of selecting and
registering the image data. The invention also improves upon earlier
methods of image viewing by adjusting to the user's line of sight while in
a dynamic field of view. FIG. 1 is a block diagram of an imaging system 2
for displaying an image of an object 10 according to a preferred
embodiment of this invention. A lead library 12 and a follow library 14 of
images of the object 10 obtained by two different modalities communicate
with a processing means 16. The imaging modality of either library could
be a CT scan, an MRI scan, a sonigram, an angiogram, video or any other
imaging technique known in the art. Each library contains image data
relating to the object.
Most preferably, at least one of the imaging devices is a device that can
view and construct an image of the interior of object 10. The images (or
data gleaned from their analysis) are stored within the libraries in an
organized and retrievable manner. The libraries may be any suitable means
of storing retrievable image data, such as, for example, electronic memory
(RAM, ROM, etc.), magnetic memory (magnetic disks or tape), or optical
memory (CD-ROM, WORM, etc.).
The processing means 16 interreferences corresponding images in image
libraries 12 and 14 to provide a map or table relating images or data in
one library to images or data in the other. The preferred interreferencing
method is described in detail below. Processing means 16 may be a
stand-alone computer such as a SGI RealityEngine (available from Silicon
Graphics, Inc.) which has been loaded with suitable software.
Alternatively, processing means 16 may be an image processor specially
designed for this particular application.
A lead imager 18 is provided to obtain an image of object 10 along a chosen
perspective or line of view. For example, if object 10 is a patient in an
operating room, lead imager 10 may be a video camera that obtains video
images of the patient along the line of sight of the attending physician,
such as a head-mounted video camera. Lead imager 18 sends its lead image
to processing means 16 which interreferences the lead image with a
corresponding follow image from follow image library 14 and transforms the
image to correspond to the lead image. The depth at which the follow image
is sliced may be controlled by a depth control 24 (such as a mouse, joy
stick, knob, or other means) to identify the depth at which the follow
image slice should be taken. The follow image (or, alternatively, a
composite image combining the lead image from lead imager 18 and the
corresponding transformed follow image from library 14) may be displayed
on display 20. Display 20 may be part of processing means 16 or it may be
an independent display.
In the preferred embodiment, object 10 has at least one fiducial marker 22.
The fiducial marker is either an inherent feature of object 10 (such as a
particular bone structure within a patient's body) or a natural or
artificial subobject attached to or otherwise associated with object 10.
The system and methods of this invention use one or more fiducial markers
to interreference the lead and follow images or to interreference lead
images acquired in real time to lead images or data in the lead image
library, as discussed in more detail below.
FIG. 2 is a flow chart showing the basic method of this invention. In the
flowchart, steps are divided into those accomplished before the start of
the surgical procedure, and those that are accomplished in real time,
i.e., during the procedure. In this example, the object of interest is a
body or a specific part of the body, such as a patient's head, and the two
imaging modalities are an MRI scan of the patient's head (the follow image
modality) and a video image of the surface of the patient's head (the lead
image modality). It should be understood, however, that the invention
could be used in a variety of environments and applications.
In the preferred embodiment, the lead and follow images are interreferenced
prior to the surgical procedure to gather information for use in real time
during the surgical procedure. Interreferencing of the lead and follow
images gathered in this pre-procedure stage is preferably performed by
maintaining common physical coordinates between the patient and the video
camera and between the patient and the MRI device. The first step of this
preferred method (indicated generally at block 30 of FIG. 2) therefore is
to mount the patient's head immovably to a holder such as a stereotactic
frame.
Next, to gather follow image information, an MRI scan of the patient's head
and stereotactic frame is taken, and the three-dimensional data (including
coordinate data relating to the patient's head and the stereotactic frame)
are processed in a conventional manner and stored in memory, such as in a
follow image library, as shown in block 34. The pre-process lead video
images of the patient's head are preferably obtained via a camera that
automatically obtains digital images at precise locations. Robotic devices
built to move instruments automatically between precise stereotactic
locations have been described by Young, R. F., et al., "Robot-aided
Surgery" and Benabid, A. L., et al., "Computer-driven Robot for
Stereotactic Neurosurgery," (in Kelly, P. J., et al., "Computers in
Stereotactic Neurosurgery," pp. 320-329, 330-342 (Blackwell Scientific
Publications, 1992)). Such devices could be used to move a camera to
appropriate lead view angles for the acquisition of the lead library. For
example, using the stereotactic frame, the video camera could move about
the head in three planes, obtaining an image every 2 mm. Each image is
stored in a lead image library along with information about the line of
view or trajectory from which the image was taken. The stereotactic frame
may be removed from the patient's head after all these images have been
obtained.
Keeping the patient's head immovably attached to the stereotactic frame
during the MRI and video image obtaining steps gives the lead (video) and
follow (MRI) image data a common coordinate system. Thus, identification
of a line of view showing a portion of a stored video image is equivalent
to identification of the corresponding line of view in the stored MRI
image. Information interreferencing the stored lead and follow images is
itself stored for use for real time imaging during the surgical procedure.
As the final step in the pre-procedure part of the method, the video lead
images are digitally analyzed to identify predefined fiducial markers. In
the preferred embodiment, the digital representation of each lead image
stored in the lead image library is segmented or broken down into
subobjects. Segmentation can be achieved by any suitable means known in
the art, such as by feature extraction, thresholding, edge detection,
Hough transforms, region growing, run-length connectivity analysis,
boundary analysis, template matching, etc. The preferred embodiment of
this invention utilizes a Canny edge detection technique, as described in
R. Lewis, "Practical Digital Image Processing" (Ellis Horwood, Ltd.,
1990). The result of the segmentation process is the division of the video
image into subobjects which have defined boundaries, shapes, and positions
within the overall image.
The Canny edge detection segmenting technique can be modified depending on
whether the image is in two or three dimensions. In this example the image
is, of course, a two-dimensional video image. Most segmentation approaches
can be adapted for use with either two-dimensional or three-dimensional
images, although most written literature concerns two-dimensional image
segmentation. One method by which a two-dimensional approach can be
adapted for the segmentation of a three-dimensional object is to run the
two-dimensional segmentation program on each two-dimensional slice of the
series that represents the three-dimensional structure. Subsequent
interpolation of each corresponding part of the slices will result in a
three-dimensional image containing three-dimensional segmented objects.
To help resolve the difficulties in segmenting low-contrast points in
images (particularly medical images), much effort in the field is being
devoted to the development of new segmentation techniques. Particularly
likely to be useful in the future are those statistical segmentation
techniques that assign to each point a certain degree of probability as to
whether or not it is a part of a given segmented object. That probability
is based upon a variety of factors including pixel intensity and location
with respect to other pixels of given qualities. Once probabilities of
each pixel have been determined, assessments can be made of the pixels as
a group, and segmentation can be achieved with improved accuracy. Using
such techniques, segmentation of a unified three-dimensional file is
preferable to performing a segmentation on a series of two-dimensional
images, then combining them, since the three-dimensional file provides
more points of reference when making a statistic-based segmentation
decision. Fuzzy logic techniques may also be used, such as those described
by Rosenfeld, A., "The fuzzy geometry of image subsets," (in Bezdek, J.
C., et al., "Fuzzy Models for Pattern Recognition," pp. 340-346 (IEEE
Press 1991)).
The final part of this image analysis step is to classify the subobjects.
Classification is accomplished by means well known in the art. A wide
variety of image classification methods are described in a robust
literature, including those based on statistical, fuzzy, relational, and
feature-based models. Using a feature-based model, feature extraction is
performed on a segmented or unsegmented image. If there is a match between
the qualities of the features and those qualities previously assigned in
the class definition, the object is classified as being of that type.
Class types can describe distinct anatomic structures, and in the case of
this invention, distinct anatomic structures as they appear from distinct
points of view.
In general, the features of each segmented area of an image are compared
with a list of feature criteria that describe a fiducial marker. The
fiducial marker is preferably a unique and identifiable feature of the
object, such as surface shapes caused by particular bone or cartilage
structures within the patient's body. For example, the system could use an
eyeball as a fiducial marker by describing it as a roughly spherical
object having a diameter within a certain range of diameters and a pixel
intensity within a certain range of intensities. Alternatively, the
fiducial marker can be added to the object prior to imaging solely for the
purpose of providing a unique marker, such as a marker on the scalp. Such
a marker would typically be selected to be visible in each imaging
modality used. For example, copper sulfate capsules are visible both to
MRI and to a video camera. As yet another alternative, the stereotactic
frame used in the pre-procedure steps may be left attached to the head. In
any case, if an object can be automatically recognized, it can be
classified as a fiducial marker.
The segmentation, feature extraction and classification steps utilized by
this invention may be performed with custom software. Suitable analysis of
two-dimensional images may be done with commercially available software
such as Global Lab Image, with processing guided by a macro script.
After the images stored in the lead and follow libraries have been
interreferenced, and the fiducial markers in the lead images have been
identified, the system is ready for use in real time imaging during a
medical procedure. In this example, real time lead images of the patient's
head along the physician's line of sight are obtained through a digital
video camera mounted on the physician's head, as in block 38 of FIG. 2.
Individual video images are obtained via a framegrabber.
In the preferred embodiment, each video image is correlated in real time
with a corresponding image in the lead image library, preferably using the
digital image analysis techniques discussed above. Specifically, the lead
image is segmented, and the subobjects in the segmented lead image are
classified to identify one or more fiducial markers. Each fiducial marker
in the real time lead image is matched in position, orientation and size
with a corresponding fiducial marker in the lead image library and, thus,
to a corresponding position orientation and size in the follow image
library via the interreferencing information. The follow image is
subsequently translated, rotated in three dimensions, and scaled to match
the specifications of the selected lead view. The process of translating
and/or rotating and/or scaling the images to match each other is k | | |
